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Abstract 

The  escalating  costs  of  military  weapon  systems  are 
constantly  under  scrutiny.  In  today's  environment  of 
reduced  funding  it  is  imperative  that  an  effective  method  of 
assessing  likely  cost  growth  be  available  early  in  the 
acquisition  life  cycle,  and  especially  during  the  source 
selection  process.  This  research  sought  to  identify  a 
method  for  predicting  the  range  of  cost  growth  around  the 
most  probable  cost  estimate  generated  during  the  source 
selection  process.  With  the  assistance  of  the  Research  and 
Cost  Division  of  Aecorautical  Systems  Division,  three 
factors  w  sre  determined  to  be  major  contributors  to  cost 
growth  for  ASD  programs;  technical  risk,  configuration 
stability,  and  sc.nedule  risk.  The  data  base  consisted  of  16 
programs  from  ASD  from  1980  to  1938.  The  results  of  this 
research  provides  a  method  for  quickly  assessing  the  range 
of  potential  cost  growth  of  the  most  probable  cost  estimate; 
however,  due  to  the  small  data  set,  more  research  must  be 
conducted  to  increase  the  method's  usefulness.  .Although 
more  research  is  required,  based  on  the  data  set  used, 
configuration  stability  appears  to  be  a  more  significant 
driver  of  cost  growth  in  the  development  phase.  While, 
schedule  risk  appears  to  be  mere  significant  in  the 
production  phase. 
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ESTIMATING  POTENTIAL  COST  GROWTH 


OF  THE  MOST  PROBABLE  COST  ESTIMATE 


I.  Introduction 


General  Issue 

The  escalating  coats  of  military  weapon  systems  have 
been  under  constant  scrutiny  since  the  late  3ix*’ies.  Much 
of  this  attention  is  focus,  d  on  the  increase  in  cost  over 
the  originally  estimated  program  costs  (22:1).  In  today's 
environment  of  reduced  funding,  and  with  the  perception  by 
the  general  public  that  the  military  does  not  manage  or  is 
incapable  of  managing  its  money  prudently,  it  is  imperative 
that  a  method  of  determining  potential  cost  growth  be 
available  earl}'  in  the  acquisition  life  cycle.  If  such  a 
method  is  jt  employed,  the  acquisition  cost  of  weapon 
systems  will  continue  to  experience  unanticipated  cost 
growth.  Cost  growth  reduces  the  DOD's  ability  to  procure 
the  number  and  type  of  weapons  necessary  to  meet  force 
structure  requirements  (27:1).  The  Navy's  A-L2  .:\ircraft 
Program  is  the  most  recent  example  of  a  program  that 
initially  suffered  a  reduction  in  quantity  and  was 
subsequently  canceled. 

The  A-12  aircraft  was  being  develo  ed  by  the  Navy  as  a 
replacement  tor  the  Navy  A-6  Intruder  aircraft,  the  Navy's 
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primary  all-weather,  medium-attack  aircraft  (34:1).  In 
December  1989,  the  Secretary  of  Defense  (SECDEF)  directed  a 
Major  Aircraft  Review  (MAR)  of  four  major  aircraft  programs 
(34:2).  The  recommendation  after  the  MAR  was  that  the  A-12 
aircraft  program  be  continued,  but  at  reduced  procurement 
quantities  (34:2).  On  June  1,  1990,  the  contractor  team 
developing  the  A-12  advised  the  Navy  that  there  would  be  a 
slip  in  the  schedule  for  the  first  flight  and  that  the 
Engineering  and  Manufacturing  Development  (EMD)  effort  would 
overrun  the  contract  ceiling  by  an  amount  that  the 
contractor  team  could  not  absorb  (34:2),  In  October  of  1990 
the  concerns  of  the  Chairman  of  the  House  Arms  Services 
Committee,  T.es  Aspin,  led  to  a  special  review  of  the  Navy's 
A-12  program  (34:i).  The  review,  conducted  from  October 
1990  to  January  1991,  "disclosed  that  the  cost,  schedule, 
and  technical  problems  in  the  A-12  aircraft  program  were  of 
such  magnitude  that  the  continued  viability  of  the  program 
was  In  serious  doubt"  (34:i).  On  December  31,  1990,  the 
contractor  i jam  submitted  a  certified  Claim  for  Equitable 
Adjustmer  for  the  EMD  contract  (34:12).  This  claim 
proposed  a  $1.4  billion  in-^rease  in  the  EMD  target  price 
(34:12).  On  January  7,  1991,  the  SECDEF  decided  to  cancel 
the  A-12  aircraft  pr<  ram  for  default  (34:13).  Though  there 
were  several  factors  leading  to  the  cancellation  of  the  .A-12 
aircraft  program,  it  is  apparent  that  the  inability  to 
control  cost  was  a  ma  ]or  player  in  the  final  decision. 


Despite  efforts  to  control  cost  growth,  there  continues 
to  be  unforseen  increases  in  cost  from  the  most  probable 
cost  estimate  generated  at  source  selection  to  the  actual 
costs  incurred  by  the  government.  Part  of  this  problem  can 
be  linked  to  the  manner  in  which  cost  is  presented  to  the 
decision  makers.  In  his  report,  "Risk  ind  the  Right  Model," 
Lt  Colonel  John  A.  Long  recommends  providing  decision  makers 
with  a  range  of  -obable  cost  to  ensure  they  have  enough 
information  to  r  e  an  informed  dec’sion.  The  need  for  this 
additional  information  is  summarized  in  the  following 
statement  by  Long: 

Normally  decision  makers  are  presented  with  only  a 
point  or  'most  likely’  cost  estimate,  with  no 
indication  as  to  the  risk  (variability)  in  that 
estimate  .  .  .  But,  point  estimates  can  be 

misleading  and  can  lead  to  a  worse  decision  than 
had  no  estimate  at  all  been  used.  (4i:2) 

Figure  1  provide.s  a  graphical  representation  of  how  not 

providing  all  the  information  can  be  misleading.  In  Case  I, 

decision  makers  are  faced  with  no  real  decision  problem''. 

because  all  possible  costs  or  System  A  are  lower  than  System 

B.  Tn  Case  II,  there  is  the  possibility  that  the  actual 

cost  of  System  A  will  be  higher  than  System  B.  The  overlap 

in  Case  II  is  not  significant  so  System  A  is  still 

preferable  to  System  8;  however,  if  the  overlap  were 

significant  "the  point  estimate  would  no  longer  provide  a 

valid  datum  for  system  selection"  (41:3).  In  Case  III,  both 

point  estimates  are  the  same,  but  the  cost  for  distribution 

B  has  a  larger  range  or  %'ariance.  ?’'eference  toward  risk 
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must  come  into  play  in  making  the  decision  in  this  instance. 

Case  IV  presents  yet  another  problem.  The  expected  cost  of 

System  B  is  lower  than  System  A.  but  has  more  uncertainty. 

The  point  estimate  alone  would  not  provide  insight  into  the 

potential  cost  growth  of  System  B. 

■\ccording  to  Long,  the  problem  with  presenting  data  in 

the  manner  indicated  above  is  that  many  decision  makers  are 

only  interested  in  a  point  estimate  (25:42).  He  cites  four 

predominate  reasons  for  this: 

First,  presenting  more  than  a  point  estimate  may 
constitute  information  overload.  Cost  is  but  one 
input  to  the  decision  process.  Information 
presented  must  he  clear,  concise,  and  easily 
understood.  Secondly,  some  decision  makers  would 
not  understand  risk  analysis  and  its  associated 
implications.  Third,  the  possibility  of  high 
costs  would  cause  und'.ie  concern  and  adversely 
affect  the  decision.  Fourth,  ri  ik  analysis  would 
impact  the  credibility  of  the  stuay  giving  the 
impression  that  analysts  were  unwilling  to  stand 
behind  their  analyses.  (41:3) 

Long  notes  that  though  decision  makers  did  not  want  the 

range  of  expected  risk,  they  did  feel  the  cost  analysts 

should  conduct  such  analyses  for  their  own  benefit  and  in 

support  of  the  point  estimate  (41:4).  The  commander  of  the 

Aeronautical  Systems  Division  of  Air  Force  Systems  Command, 

General  Ferguson,  also  expressed  the  need  to  provide 

decision  makers  with  a  range  of  potential  cost  growth  around 

the  most  probable  cost  (MFC)  of  each  bidding  contractor 

(38)  . 
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Specific  Problem 

During  the  source  selection  process  a  teum  of 
government  cost  analysts,  often  called  the  cost  panel  or 
cost-to-government  team,  dec  lops  a  most  probable  cost  (MFC) 
estimate  for  each  contractoL  hroding  to  develop  and/c;r 
produce  the  system  or  program.  The  purpose  of  the  MPC  is  to 
protect  the  government's  interest  by  i.nsuring  a  contractor 
does  not  "buy  into"  the  contract  by  submitting  an 
unrealistically  low  bid  (20:15-21).  If  a  range  of  potential 
cost  growth  is  not  provided  around  the  MPC  point  estimate 
decision  makers  will  not  have  all  the  information  necessary 
to  make  the  best  decision.  Currently,  an  acceptable  method 
of  determining  such  cost  growth  is  not  available  to  be 
utilized  early  in  the  acquisition  life  cycle  (38). 

There  needs  to  be  a  method  of  determining  the  range  of 
potential  cost  growth  early  in  the  acquisition  of  weapon 
systems.  Cue  to  the  uncertaii.ties  inherent  early  in  the 
acquisition  life  cycle  and  the  time  constraints  in  the 
source  selection  environment,  this  method  must  make  use  of 
information  available  early  in  a  system's  life  and  be  simple 
enough  to  use  in  the  limited  time  available. 

Scope  and  Limitations 

This  research  will  concentrate  on  developing  a  method 
to  improve  the  usability  of  the  most  probable  cost  generated 
during  the  source  selection  process.  Though  cost  growth  is 
experienced  in  all  phases  of  weapon  system  acquis.itions ,  the 
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focus  of  this  research  is  on  cost  growth  from  the 

development  and  production  most  probable  cost  estimates  to 

the  current  estimate.  The  data  base  consists  of  ma^or  and 

non-major  weapon  systems  that  were  initiated  in  the  past 

decade,  1980  -  1988,  at  Aeronautical  Systems  Division.  A 

major  system  is  defined  by  DoDI  5000.2  as: 

A  combination  of  elements  that  will  r unction 
together  to  produce  the  capabilities  req:”  to 

fulfill  a  mission  need,  ...  A  system  shall  be 
con.sidered  a  ma^or  system  if  it  is  estimated  by 
the  Under  Secretary  of  Defense  V.o  require: 

a.  An  eventual  total  e.xpenditare  for 
research,  development,  test  and  evaluation,  of  more 
than  275,000  in  fiscal  year  1980  constant  dollars 
(3115,000,000  in  fiscal  year  1990  constant 

doll ars ) ,  or 

b.  An  eventual  total  expenditure  for 
procurement  of  more  than  2300.000  in  fiscal  year 
1980  constant  dollars  ($540,000,000  in  fiscal  year 
1990  constant  dollars).  03:3) 

There  are  a  number  of  factors  and  events  inherent  in 
the  development,  production  and  fielding  of  a  weapon  system. 
An  attempt  to  capture  all  of  these  factors  is  beyond  the 
scope  of  this  research.  The  data  set  is  limited  to  these 
prograiTis  initiated  at  .Aeronautical  Systems  Division  and  may 
not  be  applicable  across  commands. 


Research  Ob~:ective 

The  objective  of  this  research  is  to  aevelop  a  method 
for  predicting  a  realistic  range  of  potential  cost  grcw*'h 
around  the  Government's  most  probable  rest  estimat'?. 


Inveatiaati ve  Questions 


The  following  investigative  questions  will  be  asked; 

1.  What  are  the  ma^or  sources  of  cost  growth  in  the 
development  uf  weapon  systems? 

2.  How  can  these  sources  of  major  cost  growth  be  used 
to  help  predict  potential  cost  growth  of  weapon  systems. 

3.  How  can  this  information  be  used  to  estimate  a 
range  of  potential  cost  growth  around  the  most  probable  cost 
estimate  developed  during  the  source  selection  process? 

Summary 

Cost  growth  continues  to  be  a  problem  in  weapon  systems 
acquisition.  Providing  a  range  of  probable  cort  growth 
around  the  most  probable  cost  estimate  (MPCl  would  greatly 
enhance  the  value  of  the  MFC  to  decision  makers. 

In  the  follovjing  chapters,  cost  growth  and  the  reasons 
for  cost  grouch  will  be  explored.  Chapter  II  consists  of  a 
review  of  the  literature  concerning  the  system  acquisition 
process,  the  source  selection  process,  estimating 
techniques,  and  cost  growth  in  weapon  system  acquisitions. 
Chapter  III  highlights  the  methodology  used  tor  data 
collection  and  developing  the  method  for  predicting  a  range 
of  ccst  around  the  MFC.  In  Chapter  IV,  findings  and 
analysis  of  data  collected  in  Chapter  III  are  discussed, 
followed  by  conclusions  and  recommendations  for  further 
research  in  Chapter  V. 
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Literature  Review 


II  . 


Chapter  Overview 

Cost  growth  is  not  uncommon  in  the  acquisition  of 
weapons  systems  and  is  often  addressed  in  the  literature. 
Nonetheless,  there  is  still  confusion  over  what  causes  cost 
growth  and  whether  or  not  it  can  be  controlled.  This 
chapter  begins  with  a  discussion  of  the  acquisition  and 
source  selection  processes  and  the  role  of  cost  estimating 
in  these  processes.  Followed  by  a  discussion  on  cost 
growth,  its  causes,  trends,  and  some  initiatives  aimed  at 
curtailing  the  problem. 

The  literature  search  was  conducted  using  Dissertation 
Abstracts;  on-line  search  capabilities  at  the  Air  Force 
Institute  of  Technology,  University  of  Dayton,  and  Wright 
State  University  libraries;  a  Defense  Technical  I.nformaticn 
Center  (DTIC)  search  of  cost  growth  covering  the  last  twenty 
ye  \rs ;  and  research  the  Aeronautical  Systems  Division 
Cost  and  Publications  Libraries. 

The  Systems  Acquisition  Process 

According  to  Department  of  Defense  Instruction  (DoDI) 
5000.2,  Defense  Acquisition  Management  Policies  and 
Pr  icedur  is,  an  acquisition  program  is  "a  directed,  funded 
effort  that  is  designed  to  provide  a  new  or  improved 
materiel  capability  in  response  to  a  validated  need''  (33:21. 
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The  system  acquisition  process  is  the  process  by  which  these 
programs  are  acqu.i.red.  There  are  five  major  decision  points 
and  five  distinct  though  s-'metimes  overlapping,  phases 
that  provide  the  basis  for  "comprehensive  mauagement  and  the 
progressive  decisionmaking  associated  with  program 
maturation"  (33:11)  The  Milestones  are  Milestone  0, 

Concept  Studies  Approval;  Milestone  I,  Concept  Demonstration 
Approval;  Milestone  II,  Development  Approval;  Milestone  III, 
Production  Approval;  and  Milestone  IV,  Ma^or  Modification 
Approval.  The  phases  are  Concept  Exploration  and 
Definition,  Demonstration  and  Validation,  Engineering  and 
Manufacturing  Development  (SMD),  Production  and  Deployment, 
and  Operations  and  Support  (O&S). 

A  primary  goal  in  developing  an  acquisition  strategy  is 
to  minimize  the  time  it  takes  to  satisfy  an  identified  need 
consistent  with  common  sense,  sound  business  practice,  and 
the  provisions  of  DoD  Directive  5000.1,  and  DoDI  5000.2 
(33:16).  With  this  goal  in  mind,  DoDI  5000.2  makes 
provi.sions  for  tailoring  the  acquisition  process  -.o  fit  the 
needs  of  the  .specific  program  or  system  being  acquired 
(33:16).  Tailoring  must  be  based  on  an  objective  assessment 
of  the  program's  status  and  risk  and  cannot  eliminate  core 
activities  which  must  be  accomplished  for  every  acquisition 
program  (33:16). 

These  core  activities  establish  and  document  the 
threat  and  operational  requirements  affordability, 
the  acquisition  strategy  and  program  baseline, 
cost  and  operational  effectiveness,  production 
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readiness  and  supportahility  and  developmental 
and  operational  test  ng .  (3.1:16) 

In  the  following  paragraphs  uhe  criteria  used  to  determine 

decision  ■  uthority  of  the  five  milestones  and  the  five 

phases  of  the  acquisition  process  are  discussed. 

Acquisition  Milestones  The  five  mi  estones  represent 
the  points  where  the  decisions  are  made  on  the  future  of  a 
program.  The  level  at  which  these  decisionr  are  made 
depends  on  the  classification  of  the  program.  All 
acquisition  programs,  with  the  exception  of  highly  sensitive 
classified  programs,  are  pieced  into  one  of  four  categories 
(33:12) . 


Milestone  Decision  Authority.  Acquisition 
Category  I  programs,  are  major  system  acquisitions  that  hav ; 
"statutorily  imposed  acquisition  strategy,  execution,  and 
reporting  requirements'*  (33:12).  Programs  in  Acquisition 
Category  I  are  estimated  to  have  an  eventual  expenditure  for 
research,  development,  test  and  evaluation  of  r'.ore  than  $2C0 
million  in  fiscal  year  1980  dollars  or  more  than  SI  bil’ion 
:-n  fiscal  year  1980  dollars  for  procurement  (33:13).  The 
Under  Secretary  further  designates  Acquisition  Category  I 
programs  as  Acquisition  Category  ID,  requiring  decisions  by 
the  Under  Secretary,  or  Acquisition  Category  IC,  requiring 
decisions  by  the  cognisant  DcD  Component  Head  (33:13). 
Acquisition  Category  II  programs  are  major  systems  that  do 
not  meet  the  criteria  for  Category  I,  but  have  unique 
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statutorily  imposed  requirements  in  the  test  and  evaluation 
area  (33:13).  They  may  also  have  unique  statutorily  imposed 
requirements  in  other  areas  such  as  Defense  Enterprise 
Programs  and  multiyear  procurement.  The  milestone  decision 
authority  for  Acquisition  Category  II  cannot  be  delegated 
lower  than  the  DoD  Component  Acquisition  Executive  (33:14). 
Acquisition  Category  III  programs  are  those  not  meeting  the 
criteria  for  Category  I  and  II,  that  have  been  designated 
Category  III  by  the  DoD  Component  Acquisition  Executive 
(33:13).  The  decision  for  Category  III  programs  may  be 
delegated  by  the  DoD  Component  Acquisition  Executives  to  the 
lowest  level  deemed  appropriate  within  their  ’  'spectiva 
organizations  (33:13).  Category  IV  programs  are  all  other 
pcograms,  non-major,  which  may  be  delegated  by  the  DoD 
CoTiiponent  Acquisition  Executive  to  the  lowest  appropriate 
level  (33:13). 

Milestones  0,  Concept  Studies  Approval.  At  this 
milestone  the  decision  authorities  analyze  the  .Mission  Need 
Statement  (MNS)  to  determine  if  the  need  identified  warrants 
the  initiation  of  study  efforts  of  alternative  concepts  to 
satisfy  the  need  (33:27).  Based  on  this  analysis,  they 
decide  whether  or  not  to  move  into  the  Concept  E.xploration 
and  Definition  Phase  (33:24). 

Ml  ^  es t ones  I  ,  Concept  Demonstration  .Approva  1  .  .At 
this  decision  milestone,  deci  '.ion  authorities  assess  the 
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affordability  of  a  new  acquisition  program  (33:31).  A 
favorable  decision  establishes  a  new  program  and  a  Concept 
Baseline;  initial  program  cost,  schedule,  and  performance 
objectives,  and  authorizes  entry  into  Phase  I  (33:31). 

Milestone  II,  Development  Approval.  At  this 
milestcne,  decision  authorities  meet  to  determine  if  the 
results  of  Phase  I,  Demonstration  and  Validation,  warrant 
continuation  into  the  next  phase. 


Milestone  III.  Production  Approval.  A  favorable 
decision  at  this  phase  represents  a  commitment  to  build, 
deploy,  and  support  the  system  (33:44).  Decision 
authorities  must  determine  if  the  results  cf  the  Engineering 


warrant  continuation  of  the  program  and  establish  a 
Production  Baseline.  This  baseline  contains  refined  program 
cost,  schedule,  and  performance  objectives  of  the  given 
program. 


Milestone  IV,  .Major  Modification  Approval.  This 

milestone  is  conducted  as  required.  A  major  modification  is 

defined  as  a  program  that  meets  the  criteria  of  Acquisition 

Category  I  or  II  or  is  designated  ar,  such  by  the  decision 

authority  (  33:49).  The  intent,  of  this  milestone  is: 

To  ensure  that  all  reasonable  alternatives  are 
thoroughly  examined  prior  to  committing  to  a  ma]or 
modification  or  upgrade  program  for  a  system  that 
13  still  being  produced  (33:49). 
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Phases  of  the  acquisition  Process.  The  milestones. 


discussed  above,  link  the  phases  of  the  acquisition  process. 
Though  these  phases  are  distinct,  there  may  be  some  overlap 
between  phases. 

Phase  Q.  Concept  Exploration  and  Definition.  The 
objectives  of  this  phase  are  to:  (1)  explore  various 
alternatives  that  can  satisfy  the  documented  mission  need 
(this  need  is  normally  defined  by  the  operational  command, 
but  can  be  defined  by  other  sources  such  as  contractors, 
military  groups,  and  research  organizations  such  as  The  RAND 
Corporation  and  The  MITRE  Corporation)  (39:11);  (2)  define 
the  most  promising  concept (s);  (3)  Develop  analyses  and 
information  identifying  high  risk  areas  and  risk  management 
approaches  to  support  the  Milestone  I  decision;  and  (4) 
develop  a  proposed  acquisition  strategy  and  initial  program 
objectives  for  cost,  schedule,  and  performance  for  the  most 
promising  system  concept(3)  (33:29).  This  phase  provides 
the  basis  for  assessing  the  relative  merits  of  the  concepts 
at  Milestone  I  (33:23). 

Phase  I,  Demonstration  and  Validation.  The 
Qb':ectives  of  this  phase  are  to:  (I)  better  define  the 

critical  design  ch  racteristics  and  e.xpected  capabilities  of 
the  system  concept  or  concepts  identified  in  the  previous 
phase;  (2)  demonstrate  that  the  technologies  can  be 
incorporated  into  the  system  design  with  some  degree  of 
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confidence;  (3)  uevelop  analyses  that  are  needed  in  order  to 
make  the  Milestone  II  decision;  (4)  assessing  whether  the 
most  promising  concept  designs  will  operate  in  the  intended 
operational  environment  and  proving  that  these  designs  are 
understood  and  attainable;  and  (5)  establish  a  proposed 
development  baseline  containing  refined  program  cost, 
schedule  and  performance  objectives  (33:35).  Much  of  the 
design  analysis  in  this  phase  is  accomplished  by  the 
contractor  or  contractors  who  desire  to  be  awarded  the 
development  and  production  contracts  (37:5).  In  cases  where 
there  are  two  or  more  contractors,  each  contractor  performs 
independent  analysis  and  design  studies  and  arrives  at  a 
design  proposal ,  which  is  then  evaluated  in  the  source- 
SclcCwioji  piTGc^ss*  TKc  wCurcc  GclwCvion  ptrocf^ss  will  bs 
explained  later  in  this  chapter. 

Phase  II,  "naineering  >nd  .Manufacturing 
Development .  The  objectives  of  this  phase  are  to:  (I) 
translate  the  most  promising  design  approach  developed 
during  the  Demonstration  and  Validation  Phase  into  a  stable, 
producible  and  cost  effective  system- - thi.s  design  should  be 
refined  in  terms  of  financial  and  technical  risk  associated 
with  long-term  production  (37:7);  (2)  validate  the 
manufacturing  or  production  process;  and  (3)  demonstrate 
the  system  capabilities  by  testing  to  determine  if  the 
system  meets  contract  specification  and  performance 
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requirements,  and  to  ascertain  whether  the  system  satisfies 
the  mission  need  (33:43). 

Phase  III,  Production  and  Deployment.  The 
objectives  of  this  phase  ate  to  (L)  establish  a  stable  and 
efficient  production  and  support  base;  (2)  achieve  an 
operational  capability  that  satisfies  the  mission  need,  and 
(3)  conduct  follow-on  operation  and  production  verification 
tasting  to  confirm  and  monitor  performance  and  quality  of 
the  system  (33:48).  Support  plans  will  be  implemented  in 
this  phase  to  ensure  support  resources  are  acquired  and 
deployed  with  the  system. 

Phage  TV  Operatinns  and  Support .  Though 
operation  and  support  (OSS)  have  long  been  a  part  oi  the 
acquisition  life  cycle,  it  has  only  recently  been  singled 
out  as  its  own  phase.  This  was  done  to  emphasize  the 
importance  of  operations  and  support  in  the  planning  phase. 
The  OSS  phase  begins  after  the  initial  systems  have  been 
fielded  and  overlaps  the  production  and  deployment  phase 
(33:5i),  The  beginning  of  this  phase  is  marked  by  either 
the  declaration  of  an  operational  capability  or  the 
transition  of  management  responsibility  from  the  developer 
to  the  maintainer  (33:51).  The  primary  ob:]ectives  of  this 
phase  are  to  (1)  ■  asure  the  fielded  system  continues  to 

provide  the  capabilities  required  to  meet  the  identified 
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mission  need  and  (2)  identify  shortcomings  or  deficiencies 
that  must  be  corrected  to  improve  performance  (33:bi). 

The  Role  of  Cost  in  the  Systems  Requisition  Process. 

One  underlying  theme  prevails  throughout  the  acquisition 
process;  this  theme  is  cost .  What  are  the  costs  associated 
with  this  system  and  are  the  resources  available  to  support 
the  system? 

In  the  early  days  of  a  program’s  life  cycle  a  great 
deal  of  information  crucial  to  the  future  of  the  program  is 
sought  by  the  decision  makers  (20:1-9).  Precise  cost 
estimates  are  sought,  but  programs  may  not  have  been  defined 
well  enough  to  make  precision  possible  (20:1-9). 

Decisions  made  early  in  the  acquisition  life  cycle  of  a 
program  have  great  influence  over  the  program’s  content, 
configuration,  and  cost.  Unfortunately,  ther  is  little 
information  early  in  the  program  that  can  assist  in  making 
these  important  program  decisions.  The  situation  cost 
analysts  and  decision  makers  are  faced  with  is  depicted  in 
Figure  3,  taken  from  The  AFSC  Cost  Estimating  Handbook.  .As 
time  passes,  more  information  is  known  about  the  system  and 
cost  estimates  are  likely  to  be  more  accurate,  but  the 
estimates  made  during  periods  of  certainty  have  less  impact 
on  program  configuration  and  content,  and  subsequently,  less 


impact  on  cost  . 


HIGH 


Figure  2.  Milestone  I  Dilemma 

Planning  Estimate.  Cost  information  is  required 
in  the  system  acquisition  process  as  early  as  the  Concept 
Exploration  and  Definition  Phase.  During  this  phase  one  of 
the  objectives  is  to  develop  the  initial  program  objective 
for  cost.  This  estimate,  developed  by  government  cost 
analysts,  is  known  as  the  planning  estimate.  The  planning 
estimate  is  conducted  very  early  in  the  conceptual  phase  of 
the  acquisition  process  (22:8)  and  is  the  baseline  estimate 
for  technical  and  operational  characteristics,  schedule 
milestones  and  program  acquisition  cost  (32:2-2K 
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Most  Probable  Cost  and  uevel roment  Estimate. 

During  the  Demonstration  and  Validation  Phase  two  cost 
estimates  are  of  importance;  the  most  probable  cost  (MFC) 
estimate  and  the  development  estimate  (DE).  "The  MPC 
represents  the  Government's  estimate  for  each  competing 
bidder  in  the  source  selection  environment"  (20:2-15)  and 
should  capture  any  peculiarities  associated  with  each 
offeror's  proposal  (20:15-23).  The  MPC  is  conducted  by  a 
team  of  cost  analysts  ’'<nown  as  the  Cost- to-Government  Team 
or  the  Cost  Panel.  The  basic  consideration  of  the  MPC 
estimate  is  "determining  if  the  offeror's  proposed  costs  are 
commensurate  with  the  technical  effort  proposed..."  (2:52). 
This  estimate  is  used  by  the  Source-Selection  Authority  to 
assist  in  choosing  the  winning  contractor  and  often  becomes 
the  only  meaningful  measure  of  the  realism  of  the 
contractor's  cost  proposal  (20:2-15).  The  development 
estimate  (DE),  is  the  baseline  "estimate  of  technical  and 
operational  characteristics,  schedule  milestones,  program 
acquisition  cost  (by  appropriation),  and  annual  production 
rates"  (34:2-1)  The  DE  is  prepared  toward  the  end  of  the 
Demonstration  and  Validation  Phase,  and  usually  approximates 
the  target  price  of  the  contract  (22:8).  This  estimate 
normally  serves  as  the  baseline  to  which  all  program  changes 
are  compared  (43:6). 

Current  Estimate.  The  current  estimate  is  the  DoD 
Component's  latest  forecast  of  the  system's  final  costs. 
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technical  and  operational  characteristic?,  schedule 
milestones,  and  annual  production  rates  (32:2-1).  This 
estimate  is  continually  updated  throughout  the  life  of  the 
program  (43:6). 

Though  there  are  only  four  estimates  mentioned  above, 
many  other  cost  estimates  are  concucted,  updated  and  revised 
throughout  the  weapon  system  acquisition  process.  The  list 
includes  such  estimates  as  the  should  cost  estimate  (SCE), 
the  independent  cost  analysis  (ICA),  and  the  program  cost 
estimate  (PCE)  (20:2-14,2-15). 

Cost  Estimating  and  the  Source  Selection  Process 

One  of  the  primary  objectives  of  the  source  selection 
process  is  to: 

Select  the  source  whose  proposal  has  the  highest 
degree  of  realism  and  credibility  and  whose 
performance  can  be  e.'tpected  to  best  meet  the 
government's  requirements  of  an  affordable  cost 
(31:3)  . 

In  the  following  paragraphs  the  role  of  cost  estimates 
during  the  source  selection  process  is  highlighted. 

The  Source  Selection  Process.  The  AFSC  Cost  Estimating 
Handbook  defines  four  phases  of  the  source  selection 
process : 

(1)  Pre-evaluation  phase 

(2)  Initial  evaluation  phase 

(3)  Intermediate  evaluation  phase 

(4)  Final  evaluation  phase 
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These  phases  will  be  defined  below  with  a  discussion  i.  the 
role  of  the  cost  estimates  in  e.^ch  phase. 


Pre-evaluation  Phase.  This  phase  typica.ly  begins 
with  the  submission  of  a  Justification  for  Major  Systems  Mew 
Start,  which  leads  to  the  issuance  or  one  or  more  Program 
Management  Directives  (PMD)  (20:15-3).  Upon  re’ ease  of  the 
PMD  the  chairperson  of  the  Source  election  Advisory 
Commitree  notifies  all  appropriate  Air  Force  Corranands  and 
potential  offerors  that  source  selection  action  is  in 
progress  (31:13).  The  Source  selection  Plan  (SSP)  and  the 
solicitation  are  two  of  the  most  important  documents 
prepared  during  this  phase  (20:15-4). 


plan  ^or 
analysis 
sources 
contents 
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organizing  and  conducting  the  evaluation,  the 
of  proposals,  and  the  selection  of  the  source  or 
to  satisfy  the  system  requirements  (36:15-4).  The 
of  tais  document  are  outlined  below  (20:15-4): 

(1)  Source  selection  organization 

(2)  Screening  criteria  to  determine  sources  to 
sol  ici tations 

(3)  Evaluation  criteria 

(4)  Acquis-  on  strategy 

(5)  Schedule  ot  events 

(o',  Evaluation  procedures  to  be  used  to  evaluate 


proposal  3 
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The  section  on  evalu.'tion  procedures  is  important  to  the 
cost  analyst  and  should  include  inputs  from  cost  analysts  in 
its  devel  opmei't .  This  section  includes  procedures  and  plans 
for  formulating  the  MPC  to  the  government;  developing  the 
independent  cost  analyses,  life  cycle  cost  estimates,  and 
design-cost-goals;  identifying  items  requiring  special 
attention  and  non-quantif iaLI e  cost  risks;  and  evaluating 
offerors  proposals  (3L:L2). 


Sol icitation.  The  solicitation  is  the  final 
step  in  the  pre-evaluation  phase  (20:15-5).  The 
solicitation  for  bid  is  commonly  referred  to  as  the  Request 
for  Proposal  (RFP)  (20:15-5).  The  RFP  contains  a 
comprehensive  description  of  the  work  to  be  performed, 
instructions  on  how  to  prepare  proposals,  evaluation 
criteria,  and  the  basis  for  award  (20:15-5).  Phase  II, 
Initial  Evaluation  begins  with  receipt  of  the  proposals 
( 20 : 15-7 ) . 


Initial  Eva'-U.-tion  Phase.  During  the  Init’a! 
Evaluation  Phase  the  cost  analyst  first  re\'iews  the 
proposals  to  ensure  all  the  cost  data  formats  required  by 
the  RFP  have  been  submitted.  If  all  the  required  formats 
are  not  provided,  the  contractor  is  given  the  opportunity  to 
submit  the  requested  data.  Once  this  is  complete,  the  ne.xt 
step  IS  the  initial  evaluation  itself. 


Initial  evaluation  of  each  proposal  lo  reviewed 
against  criteria  set  forward  in  the  SSP .  The  proposal  is 
checked  for  reasonableness,  realism,  and  completeness. 
Reasonableness  is  used  to  evaluate  the  acceptability  of  the 
bidder's  methodology  (2:52).  Realism  is  used  to  evaluate 
the  compatibility  of  costs  with  the  scope  of  the  proposal, 
ensuring  that  the  estimate  is  neither  excessive  nor 
insufficient  for  the  effort  to  be  accomplished  (20:15:19). 
The  estimate  also  must  be  consistent  with  the  requirements 
of  the  Request  for  Proposal  (RFP),  Statement  of  Work  (SOW), 
and  ".  .  .  all  information  required  by  these,  including 

appropriate  program  ground  rules,  schedules,  and 
constraints.  .  ."  (2:53).  Completeness  is  used  to  evaluate 

the  "responsiveness  of  the  offeror  in  providing  all  RFP 
requirements,  statement-of -work  (SOW)  items,  and 
traceability  of  the  estimates"  (20:15-20). 

The  initial  MFC  estimates  are  conducted  parallel  with 
the  reasonableness,  realism  and  compleeness  evaluations. 
This  estimate  is  refined  and  updated  as  more  information 
becomes  available  and  does  not  become  final  until  the  Best 
and  Final  Offer  (BAFC)  is  received  and  evaluated  (20:15-3). 

One  of  the  primary  goals  of  the  initial  evaluation 
phase  is  to  prepare  for  discussions  with  the  offerors 
(20:15-9).  The  initial  evaluation  phase  is  iollowed  by  the 
Intermediate  phase  (20:15-9). 
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Intermediate  Phase.  The  primary  task  of  the  cos 


panel  in  the  intermediate  phase  is  supporting  discussions 
with  offerors  and  adjusting  the  initial  MPC  estirnati  s  and 
evaluations  as  the  offerov’s  intentions  become  more  clearly 
defined  (20:15-9).  The  purpose  of  the  contractor 
discussions  is  for  each  side,  both  the  government  and 
contractors,  to  become  more  knowledge  about  the  other's 
intentions  and  clarify  outstanding  issues  (20:15-9).  At  the 
end  of  this  phase  the  offerors'  designs  are  usually  firm  and 
the  MPC  is  completed  based  on  this  stabilized  design 
( 20 : 15-9) . 


Final  Phase.  During  the  final  phase,  the  MPC 
estimates  and  evaluations  have  to  be  finalized  (20:15-10). 
The  evaluations  are  presented  to  the  Source  Selection 
Advisory  Council  (SSAC),  who  then  rates  the  proposals  and 
presents  their  findings  to  the  Source  Selection  .Authority 
(SSA).  The  SSA  then  makes  the  source  selection  decision, 
and  the  contracting  officer  awards  the  contract(j)  to  the 
winner(s)  (20:15:14). 

Estimating  Techniques 

The  cost  estimating  tasks  ,*:<p  )  ■  ned  during  the 

acquisition  and  source  selection  prcce -.ses  are  accomplished 
using  various  techniques  av -.liable  to  the  cost  analyst. 

Four  nt  these  techniques  are  explaint-d  in  the  following 
paragra-phs  . 
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Parametric  Approach.  Parametric  estimating  is 
accomplished  by  correlating  design  parameters  to  historical 
costs  with  the  use  of  regression  analysis.  It  often  uses 
cost  estimating  relationships  (CERs)  to  help  predict  future 
weapon  systems  costs  (40:3).  CERs  relats  costs  as  the 
dependent  variable  to  one  or  more  independent  variables. 

The  parametric  technique  is  used  most  often  when  there  is 
limited  program  and  technical  definition  (20:3-21). 

Analogy  Approach.  The  "analogous"  or  "comparative" 
method  is  derived  by  choosing  analogous  programs  that  have 
previously  been  completed  and  for  which  cost  data  is 
available  (39:26).  The  cost  analyst  derives  estimates  for 
the  new  program  by  adjusting  for  complexity,  technical,  or 
physical  differences.  This  method  is  normally  chosen  early 
in  the  cycle  when  there  is  insufficient  actual  cost  data  to 
use  as  a  basis  for  a  detailed  approach;  but  there  is  a 
sufficient  amount  of  program  and  technical  definition  based 
on  study  results  and  test  data  (20:3-24). 


The  Grass  Roots  Approach.  The  grass  roots  method,  also 
referred  to  as  the  "detailed"  or  "engineering"  estimate, 
involves  br ?aking  down  the  project  into  detailed  work 
segments  that  can  be  individually  estimated  at  an  assumed 
high  level  of  accuracy.  The  grass  roots  estimate  would 
normally  be  utilised  during  the  production  phase  of  the 
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program  cycle  when  program  conf iguj’ation  has  stabilized 
(  20:3-25)  . 


Expert  Opinion  Cost  Estimates.  This  is  a  subjective 
technique  i-hat  is  dilliculL  Lu  analyze  and  :3abs  LanLxaLc 
(39:31).  This  method  is  best  applied  on  new  products  that 
are  beyond  the  current  state  of  the  art  (39:31). 

The  estimating  techniques  above  are  not  necessarily 
used  exclus.ively .  In  any  given  program  one  two,  three  or 
all  four  or  the  techniques  will  be  used  in  estimating  the 
cost  of  the  program.  The  estimator  chooses  and  combines 
estimating  methodologies  based  on  the  estimating  task  to  be 
accomplished  (20:3-28). 

Cost  Grow  ji 

The  AFSC  Cost  Estimating  Handbook  defines  cost  growth 
as,  "A  term  related  to  the  net  change  of  an  estimated  or 
actual  amount  over  a  base  cost  figure  previously 
established"  (20:A-22).  In  his  paper,  "Cost  Growth  and  the 
Use  of  Competitive  Acquisition  Strategies,"  Frederick  Biery 
defines  cost  growth  as,  "The  difference  between  the  actual 
costs  (or  the  most  current  estimate  of  actual  costs)  and  the 
estimate  at  the  start  of  the  system's  development" 

(23:1).  In  the  Congressional  Budget  Office’s  report,  "Cost 
Growth  in  Weapon  Systems:  Recent  Experience  and  Possible 
•Remedies,"  cost  growth  refers  to  the  tendency  for  the  unit 
cost  of  a  system  to  i.ncrease  during  the  course  or  the 
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acquisition  process  (33:2).  In  his  doctoral  dissertation. 
Major  Richard  Sapp  states  that  because  the  term  cost  growth 
is  not  standardized  it  has  led  to  "misunderstandings  and 
communication  difficulties,  as  well  as  making  comparisons  of 
available  research  work  and  studies  difficult"  (46:15).  He 
notes  that  the  term  is  often  used  interchangeably  with  the 
terms  contract  growth,  cost  estimate  growth,  program  cost 
growth,  price  increase,  etc.,  but  was  originally  intern  ad 
to  refer  to  increases  in  costs  due  to  influences  beyond  the 
control  of  the  weapon  system  acquisition  program  managers 
(46:15).  Cost  growth,  in  the  context  of  this  research,  is 
consistent  with  Biary's  use  of  the  term,  the  difference 
between  the  actual  cost  (or  the  most  current  estimate  of 
actual  costs)  and  the  estimate  made  at  the  start  of  the 
system's  development.  The  term  "actual  cost"  and  "most 
current  estimate  of  actual  costs"  will  be  used 
interchangeabl y . 

Despite  the  different  uses  of  the  term  cost  growth 
presented  in  the  literature,  the  consensus  is  that  cost 
growth  is  inevitable.  Though  cost  growth  is  widely  accepted 
as  unavoidable  due  to  uncertainty  in  predicting  the  future, 
cost  estimates  are  expected  to  reasonably  predict  the  future 
and  cost  growth  is  expected  to  be  kept  unde :  control.  In 
order  to  control  or  solve  a  problem;  first,  the  causes  of 
the  problem  must  be  identified  and  secondly,  measures  aimea 
at  eliminating  or  reducing  these  causes  must  be  implemented. 
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Unanticipated  cost  growth  in  weapon  system  acquisitions,  is 
complicated  by  a  myriad  of  sources  identified  as 
contributors  to  the  problem.  In  the  following  section  some 
of  the  contributors  to  cost  growth  identified  in  the 
literature  will  be  discussed. 

Reasons  for  Cose  Growth.  A  1982  House  Arms  Services 
Committee  Special  Panel  on  Defense  Procurement  Procedures 
identifi  d  poor  cost  estimates,  program  stretch-outs, 
changes  in  weapon  systems  specifications,  inadequate 
budgeting,  unrealistic  inflation  estimates,  and  lack  of 
competition  as  controllable  factors  (as  opposed  to 
uncontrollable  factors  such  as  inflation)  that  cause  cost 
growth  (27:2;.  in  his  article,  "Military  Systems 
Procurement,"  John  Adam  attributes  cost  growth  to 
unrealistic  performance  and  cost  schedules  deliberately 
provided  by  contractors  and  accepted  by  program  management 
in  order  to  get  program  contracts  (1:30).  A  1.979  Rand 
Corporation  report,  conducted  by  Edmund  Dews,  et  al . , 
identified  schedule  slippage,  engineeri.ng  changes, 
estimating  errors ,  and  changes  in  the  support  area  as  the 
major  sources  of  cost  growth  in  the  1970s  (35:91).  The 
major  reason  given  for  schedule  slippa  in  this  Rand  report 
was  inadequate  funding  levels,  while  the  engineering 
variances  were  attributed  to  unexpected  technical 
difficulties  and  changes  in  the  performance  requirements  of 
the  system  (35:92-93).  The  estimation  errors  were  found  to 
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be  the  result  of  mistakes  in  estimator  judgement  such  as 

inappropriate  analogies  or  estimating  relationships  and 

omission  of  costly  system  elements  such  as  training  or  depot 

equipment  (35:93).  Yet  another  approach  to  analyzing  cost 

growth  was  taken  by  Major  Martin  D.  Martin.  In  his  doctoral 

dissertation,  Martin  separated  the  causes  for  cost  growth 

into  two  categories;  pre-activation  and  r.cti vation . 

Pre-activation  refers  to  the  time  period  between 
technical  and  cost  proposal  preparation  and  the 
time  that  tne  contract  is  signed  by  both  parties. 
Activation  refers  to  the  contract  administration 
and  closure.  (28:87) 

In  the  pre-activation  area,  Martin  includes  such  items  as 
lack  of  competition,  contractor  underpricing,  variability  in 
past  cost  data,  concurrency  of  research  and  development  with 
production,  extraneous  design  requirements,  faulty  technical 
planning,  inadequate  task  definition,  reliability  problems, 
budgetary  constraints,  and  communication  problems.  In  the 
activation  period  he  included  such  items  as  inflation, 
quantity  variances,  lack  of  cost  control,  inadequate 
management  controls,  program  stretch-outs,  engineering 
changes,  and  technological  obsolescence  (42:39).  3y 
observing  cost  growth  in  two  separate  phases  Ma^or  Martin 
emphasized  the  need  to  understand  not  only  why  cost  growth 
occurs,  but  also  where  it  occurs. 

The  Congressional  Budget  Office  (C30)  determined  that 
the  bulk  of  cost  growth  is  likely  to  occur  after  the  system 
enters  the  Engineering  and  .Management  Development  ( EMD  ■ 


phase  and  before  its  full-scale  production  (47:3).  In  a 
study  conducted  by  the  Institute  for  Defense  Analysis  (IDA) 
It  was  shown  that  "achievement  of  Initial  Operational 
Capability  (IOC)  marks  the  end  of  significant  growth  in  both 
development  and  procurement  costs  for  most  systems"  (21:28). 

Measures  to  Reduce  Cost  Growth.  Cost  growth  is  not  a 
problem  unique  to  the  DoD.  In  the  General  Accounting 
Office's  (GAO's)  tabulation  of  cost  growth  in  major 
acquisitions  of  NASA,  the  DoD,  and  Federal  non-defense 
agencies  (those  costing  more  than  $50  million),  t.hey  found 
that  the  average  cost  growth  was  32  percent.  The  cost 
growth  for  defense  alone  for  the  same  period  was  79  percent, 
thus  indicating  that  the  defense  cost  growth  is  below 
average  (26:10).  Though  other  studies  have  also  shown  that 
cost  growth  experienced  by  the  DoD  is  less  than  those 
experienced  in  the  public  sector  (23:6),  cost  growth  of 
weapon  system  acquisitions  continues  to  be  if  concern  to 
Congress,  key  decision  makers,  and  the  general  public  who 
would  rather  see  funds  spent  elsewhere  (37:69).  It  would  be 
easy  to  point  to  findings  such  as  the  GAO's  report  mentioned 
above  to  try  to  negate  the  charges  of  individuai-s  who 
criticize  DoD's  poor  cost  performance  history.  As  Donald 
Chamberlain  phrases  it  in  his  article  to  the  1935  National 
Estimating  Society  Conference,  "Using  Labor  Standards  for 
Estimating" : 


We  can  search  to  find  fault  with  the  accusers  and 
attempt  to  discredit  their  allegations,  or  we  can 
be  more  positive  and  spend  our  time  searching 
intarnally  for  pcsitiva  improvement  techniques. 

(25:1) 

The  DoD  has  chosen  the  later  approach  and  over  the  years  has 
devoted  a  great  deal  of  time  and  energy  "searching 
internally  for  positive  improvement  techniques"  to  curtail 
cost  growth. 

Congressional  dissatisfaction  with  cost  overruns  and 
late  deliveries  has  given  rise  to  a  series  of  1 aws  designed 
to  improve  the  acquisition  process  and  to  identify  problems 
which  may  result  in  cost  growth  or  cost  overruns  as  soon  as 
possible  (23:1).  The  initiatives  of  David  Packard, 

Secretary  of  Defense  form  l'?S9  to  1971.,,  had  perhaps  greatest 
impact  on  acquisition  policy  (35:1).  Packard's  initiatives 
emphasized  ten  major  policy  elements  including  the  following 
cost  considerations: 

1.  Design-to-cost :  establish  a  cost  goal  as  one 
of  the  primary  program  objectives,  equal  to 
schedule  and  performance  in  importance;  design 
with  operation  and  support  costs  in  mind  as  well 
as  production  cost  (life  cycle  costing).  (35:2) 

2.  Improve  program  cost  estimates  and  provide  OSD 
[Office  of  the  Secretary  of  Defense]  with  an 
independent  source  of  such  estimates  by 
establishing  a  Cost  .Analysis  Improvement  Group 
(GAIG)  within  OSD.  (35:2) 

Some  of  the  other  initiatives  were  establishment  of  high 
Level  reviews  for  major  weapon  systems  at  important  program 
decision  points,  better  training  and  longer  tours  for 
program  managers,  and  reduction  in  development  and 
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production  concurrency  allowing  for  increased  testing  during 
the  development  cycle  (23:15).  Packard  also  noted  the  high 


level  of  uncertainty  in  development  programs  and  in  1969  it 
became  mandatory  to  include  risk  analysis  as  part  of  the 
acquisition  process  (24:1). 

Currently  Air  Force  Systems  Command  Regulation  (AF3CR) 
173-2,  requires  that  all  cost  estimates  prepared,  briefed 
and  documented  within  Air  Force  Systems  Command  (AFSC) 
convey  the  basis  of  confidence  using  a  qualitative  method. 
AFSCR  173-2,  specifies  that  the  qualitative  method  provide 
an  assessment  of  the  availability  and  quality  of  data  used 
in  preparing  the  cost  estimate  (30:2).  It  assigns  the 
estimate  to  one  of  the  five  following  categories  of 
confidence  (30:2): 

Category  I:  Actual  cost  for  significant 
production  quantities  is  available  for  system 
being  estimated. 

Category  II:  Actual  cost  for  development  or  early 
production  hardware  is  available  for  system  being 
estimated . 

Category  III:  No  actuals  available,  but  sysrem  is 
well  defined  with  good  data  for  analogous,  factor, 
parametric  estimate  or  contract  proposal. 

Category  IV:  No  actuals  available.  System 
description  is  fair  to  good.  Some  data  available 
for  analogies,  factors,  oarametncs,  etc. 

Category  V:  Uncertain  system  description.  Data 
available  for  analogies,  factors  parametrics,  etc. 
are  sparse  or  have  limited  value  to  system  being 
estimated . 

Once  classified;  however,  AFSCR  173-2  does  not  further 
e;<plain  how  these  classifications  can  be  used  to 
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determine  the  quality  af  the  estimate.  Whereas,  most  people 
would  agree  that  more  confidence  should  be  ?la  id  in  an 
estimate  developed  in  Category  I  than  one  deve  ed  in 
Category  V,  this  method  does  nothing  to  help  the  decision 
maker  or  cost  analyst  determine  how  much  confidence  should 
be  placed  in  these  estimates. 

In  addition,  to  the  qualitative  method  of  assigning 
confidence  ment-ioned  above,  AFSCR  173-2  requires  a 
quantitative  metnod  of  risk  assessment  be  accomplished  when 
applicable.  AFSCR  173-2  specifies  tnat  the  AFSC  Risk  .Model 
be  used  to  accomplish  the  quantitative  estimate  (30:2).  The 
AFSC  Risk  model  is  currently  undergoing  redesign  to  improve 
deficiencies  noted  in  The  Analytical  Sciences  Corporation's 
(TASC's)  definition  study  of  the  Risk  model.  Surveys 
conducted  by  T.ASC  during  the  definition  study  revealed  that 
users  felt  the  model  was  not  user  friendly  and  was  too  slow 
(36:3-4).  Though  the  risk  model  was  designed  to  be  used  at 
all  phases  in  the  acquisition  life  cycle  (  3  6 :  Appendi  ;<  A),  it 
IS  too  cumbersome  and  time  consuming  to  be  used  during 
development  of  the  most  probable  cost  during  the  source 
selection  process  (23).  T.ASC' s  overall  assessment  of  the 
AFSC  Risk  Model  was  that  it  is  useful ,  but  in  need  of 
improvements  i.a  order  to  gai.n  user  acceptance  (36:1’-1). 

Trends  in  Cost  Growth.  Tlie  literature  reviewed 
indicates  a  positive  trend  la  cost  growth  from  the  1  250s  ",  o 
the  early  1280s  (ZZ)  ^ 3 5 1  (4f'  (43).  In  his  analysis  of 
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cost  trends  over  three  decades,  Biery  noted  three  factors 
whose  tnfluence  might  distort  cost  growth  comparisons  across 
the  decades;  improved  documentation  of  original  cost 
estimates,  different  make-up  of  the  systems  being  used  in 
each  time  period,  and  the  length  of  time  to  field  a  system 
from  initial  estimate  (23:9-13).  Due  to  increased  emphasis 
on  cost  estimating,  the  documentation  of  cost  estimates 
improved  in  the  1970s  over  the  1950s  and  the  L960s;  thereby, 
improving  the  ability  tc  track  cost  growth  (23:9).  The 
difference  in  the  make-up  of  the  systems  being  procured 
could  have  an  impact  on  cost  growth  if  the  propensity  for 
cost  growth  depends  on  the  type  system  being  procured.  In 
the  19503  aircraft  systems  made  up  the  ma:crit7  of  the 
systems  being  acquired  (23:9).  In  the  1960s  aircraft  only 
made  up  one  third  of  the  acquisitions  and  electronics  made 
up  half  the  programs  (23:9).  The  1970s  sample  is  dominated 
by  missile  and  gun  systems  followed  by  aircraft  and  cruise 
missiles  (23:10).  Further  complicating  the  scenario,  the 
Affordable  Acquisition  Approach  Study  noted  an  increase  in 
the  development  and  pre-development  times  in  the  acquisition 
of  most  weapon  systems  (29:2-54). 

To  account  for  theses  di.;,tortiQns ,  Biery  made 
ccmpariscns  between  the  dec-ades  using  a  compound  annual 
jrowth  rate  and  comparisons  of  means  '(average  cost  gr'wtn 
'23:11).  However,  to  'nsure  the  comparisons  were  eaui table, 
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Biery  usea  the  following  formula  to  normalize  for  the 
increased  development  effort: 

CF  =  (L+R)*  (1) 

where 

CP  =  actual  cos t / est imated  cost 
R  =  the  rate  of  growth 
T  =  the  time  from  start  of  PSD 

The  results  of  Biery's  analysis  contained  in  the  tables 
below  suggests  that  there  have  oeen  improvements  in  the 
accuracy  of  weapon  system  cost  estimates  over  the  years. 
Though  he  shows  improvement  over  the  years  normalizing  for 
the  length  of  the  program,  the  lengthening  of  the 
development  cycle  has  eroded  the  effects  of  these 
improvements  in  real  terms  (23:12). 

The  samv»  downward  trend  was  observed  when  Biery 
observed  the  dispersion  or  "spread"  around  the  means  for 
each  decade.  The  dispersion  around  the  mean  is  measured  by 
the  standard  deviation  and  the  dispersion  around  the  growth 
rate  by  the  standard  error  of  the  estimate. 

Table  1 

Cost  Growth  Decade  Companst  i  (23:12) 

Average  ?’'ogram  Compound  Annual 

Decade  Age  Y -lars  ■  e_a n  ■  Ci'  own  i  ^ 

arly  19503 
arly  lO^Os 
arly  Ij^Os 


5.91  1.36  14.3 

:  .  30  1.45  ”  .  3 


Table  2 


Cost  Factor  Dispersion  By  Decade  (23:13; 


Decade 

Mean 

Standard 

Deviation 

Compound 

Annual 

Growth  Rate  % 

Standard 
Error  of  the 
Estimate 

1950s 

1  .  36 

1 . 386 

14.0 

0.028 

1960s 

1.45 

0 . 441 

7 . 8 

0.023 

1970s 

1.59 

0 . 481 

5 . 0 

0 . 007 

The  results  of  Biery's  analysis  show  that  there  has 
been  improvement  in  military  cost  forecasting  accuracy  since 
the  1950s  (23:3).  The  records  have  not  gotten  any  better  in 
actual  dollar  amount  of  overruns  because  of  the  lengthier 
forecasting  horizons  ''23:14). 

A  1979  Band  Corporation  study,  Acquisition  Policy 
Effectiveness:  Department  of  Defense  Experience  in  the 

1970s ,  showed  the  same  reduction  in  real  cost  growth  from 
the  19t'0s  to  the  1970s.  The  real  cost  growth  for  ma:;or 
weapon  systems  averaged  7  to  3  percent  annually  during  the 
1960s,  compared  with  5  to  6  percent  annually  for  ma^or 
weapon  systems  in  the  L970s  (35:56).  A  study  by  the  Defense 
Sciences  Beard  also  showed  a  decrease  in  the  cost  growth 
rate  from  the  1960s  to  the  19703  (12:58).  Though  the 
downv,-ard  trend  experienced  in  the  i970s  may  suggest  that  the 
cost  growth  problem  is  under  control,  a  1983  report  by  the 
Congressional  Budget  Office  (C30),  suggest.!  that  the  cost 
growth  trend  seems  to  be  increasing  in  the  1930s  (47:2).  In 
the  Institute  for  Defense  Analysis  document,  issues  in 


Measuring  Cost  Growth.  Tyson,  et 
policy  makers  in  the  early  19a0s 
was  a  thing  of  the  past  (48:12). 
several  factors  which  could  have 
experienced  ir  the  early  1980s: 


al . ,  noted  that  many 
speculated  that  cost  growth 
Tyson,  et  al.,  highlighted 
distorted  cost  growth 


New  Programs.  During  the  1980s,  there  was  an 
unusually  large  number  of  new  starts.  In  the 
1981~85  period  alone,  at  least  18  ma^or  programs 
went  into  FSD.  As  shown  in  a  number  of  studies, 
cost  growth  is  relatively  low  in  new  programs,  and 
tends  to  increase  as  uncertainty  decreases  and 
reality  is  faced.  (48:13) 

Program  Funding.  Programs  in  the  198Qs  were  amply 
funded,  due  to  the  defense  buildup.  Therefore, 
cost  analysts  have  less  incentive  to  be  optimistic 
about  costs  when  doing  the  development  estimate 
(48:13). 

Inflation  Dividend.  During  the  late  1970s, 
inrlatian  outran  program  funding  requirements. 
During  the  early  1980s,  the  rate  of  inflation 
declined,  and  the  DoD  (and  many  other  forecasters) 
overestimated  inflation.  This  resulted  in  program 
overfunding  in  then-year  dollar  terms.  .Again, 
this  made  it  seem  like  there  was  less  cost  growth. 
(48:13) 

Programs  Not  Included.  During  the  1980s,  there 
were  several  major  "black”  programs,  particularly 
those  involving  Stealth  technology.  These 
programs  were  not  included  in  cost  growth 
estimates,  but  the  DoD  is  currently  trying  to 
include  the  costs  even  of  sensitive  programs 
wherever  possible.  (48:13) 


Summary 

In  this  chapter  literature  search  was  conducted  to 
gain  more  insight  into  ihe  role  of  jost  estimating  in  the 
weapon  acquisition  life  cycle  and  the  source  selection 
processes  and  to  explore  the  reasons  for  cost  growth  and 
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some  of  the  methods  currently  used  to  control  it.  The 
literature  suggests  that  there  has  been  a  decrease  in  cost 
growth  from  the  1950s  t  the  19''0s.  These  decreases  can  be 
attributed  to  such  measures  as  the  Packard  Initiatives  and 
increased  competition.  Despite  positive  trends  over  the 
past  three  decades,  current  research  indicates  that  the 
trend  was  reversed  in  the  1980s. 
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III.  Methodology 


Chapter  Overview 

This  chapter  explains  the  methodology  used  to  answer 
investigative  questions  one  through  three  from  Chapter  I . 

The  data  collection  procedures  and  methods  of  analyzing  the 
data  are  discussed  along  with  an  explanation  of  how  the  data 
was  used  to  predict  the  range  of  potential  cost  growth. 

Factors  Contributing  to  Cost  Growth 

The  first  question  to  be  answered  was,  "What  are  the 
major  sources  of  cost  growth  in  the  development  of  weapon 
systems?"  A  review  of  the  literature  concerning  cost  growth 
revealed  several  factors  considered  to  be  contributors  to 
cost  growth.  /s  mentioned  in  Chapter  II,  soma  of  these 
factors  are  beyond  the  control  of  the  Defense  establishment. 
The  controllable  cost  factors  identified  in  the  literature 
reviewed  included  unrealistic  inflation  estimates,  poor  cosr 
estimates,  .ack  of  competition  among  defense  contractors, 
high-risk  system  design,  poor  management,  changes  in  weapon 
systems  specifications,  unrealistic  performance  and  cost 
schedules,  variability  m  past  cost  data,  concurrence  of  the 
development  and  production  efforts,  and  technical  advances 
(1;  (27',  (  28  )  (  35  '  (42  )  (  46)  (  47  ).  Since  the  writing  of 

some  of  the  literature,  laws  and  c.hanger  in  acquisition 
policies  have  curtailed  some  of  the  problems  identified  in 
the  literature.  For  example,  lack  of  competition  among 


contractors  is  not  one  of  the  problems  mentioned  in  the 
later  literature.  A  Rand  comparison  of  ten  systems  found 
evidence  that  competition  led  to  modest  improvements  in 
system  performance  and  on-schedule  delivery  by  contractors, 
and  had  substantially  lowered  real  cost  growth  (35:28).  To 
obtain  a  timely  analysis  of  those  factors  considered  to  be 
ma:or  contributors  of  cost  growth  the  expert  judgement  of  a 
panel  of  cost  analysts  from  Aeronautical  Systems  Division 
(ASD)  was  enlisted.  The  panel  consisted  of  the  followi.ng 
individuals  from  Aeronautical  Systems  Division,  Research  and 
Cost  Division,  Directorate  of  Cost,  Deputy  for  Financial 
Management  and  Comptroller; 

James  L.  Adams,  Cost  Analyst 

Charlie  Clark,  Cost  Analyst 

Donna  Kinlin,  Cost  Analyst 

William  H.  Kugel ,  Chief  Source  Selection 

Robert  Schwenke,  Chief  Rasearch/Methods  Branch 

Kenneth  Sullivan,  Capt,  USAF,  Cost  Analyst, 

A  two  step  process  was  used  to  identify  those  cost 
factors  deemed  to  be  significant  contributors  to 
unanticipated  cost  growth.  Step  one,  entailed  identifying 
those  factors  that  are  considered  to  be  contributors  to  cost 
growth,  based  on  the  literature  search  and  the  opinion  of 
selected  members  of  the  cost  panel.  I.n  the  second  step,  the 
factors  which  were  identified  in  step  one  were  ranked  by  the 
cost  panel  to  determine  the  three  top  factors  which 


40 


contribute  to  cost  growth.  The  members  of  the  cost  panel 
were  asked  tu  yive  the  three  factors  they  believed  to  be 
most  significant  drivers  of  cost  growth  a  rank  of  one.  The 
decision  to  rank  the  top  three  factors  as  number  one  was 
based  primarily  on  the  information  gained  through  the 
literature  search.  Throughout  the  literature  the  authors 
did  not  commit  to  any  one  factor  as  most  impor  ant,  but 
usually  considered  a  collaboration  of  factors  which  were 
most  important  for  a  given  time.  Secondly,  it  was  the 
consensus  of  the  preliminary  cost  panel  t.hat  concentrating 
on  three  or  four  factors  considered  to  be  significant 
drivers  of  cost  growth  would  be  most  beneficial  to  the 
study.  The  author  decided  on  three  factors  due  to  the 
limited  data  base  available.  All  other  factors  were  ranked 
consecutively  beginning  with  number  four. 

Data  Collection 

As  stated  in  Chapter  I,  this  research  considers  major 
and  non-major  weapon  system  acquisitions  started  after  1980 
with  at  least  three  years  of  development.  The  following 
information  was  gathered  on  each  of  the  systems: 

1.  The  most  probable  cost  (MPC)  estimate. 

2.  The  actual  development/procurement  cost  or  latest 
estimate  if  t.he  program  is  not  complete.  This  cost  will  be 
considered  the  current  estimate  whether  actuals  or  an 
estimate . 
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3.  The  technical  risk,  configuration  stability,  and 
schedule  variance. 

All  costs  were  collected  in  then  year  dollars  to  capture  the 
net  effect  of  all  cost  growth,  including  inflation.  The 
only  adjustments  made  were  for  quantity.  When  the  current 
estimate  (CE)  quantity  is  different  from  the  MPC  estimate, 
the  quantity  estimated  at  the  time  of  the  MPC  will  be  u.sed 
as  the  baseline  and  the  CE  will  be  adjusted  up  or  down 
accordingly.  For  example,  if  the  quantity  of  systems  to  be 
acquired  was  originally  projected  at  five  and  the  current 
estimate  allows  for  three,  the  current  estimate  will  be 
adjusted  to  reflect  five  systems. 

Quantity  AoiusimenL.  i^uaiiLxLy  au justments  were 
accomplished  in  two  ways.  One  method  was  employed  for 
Selected  Acquisition  Report  (SAR)  programs  whose  quantities 
were  the  same  in  the  MPC  estimate  and  the  development 
estimate,  and  another  method  for  non-SAR  programs  and  SAR 
programs  whose  MPC  and  SAR  development  estimate  quant  ties 
differ. 

SAR  Programs,  MPC  Quantity  Equals  OE  Quantity. 

For  these  programs  the  CE  was  simply  adjusted  up  or  down  by 
adding  the  dollar  amount  attributed  to  a  quantity  change 
reported  in  the  1990  SAR.  This  method  of  adjusting  SAR 
programs  is  explained  in  the  Rand  i9?9  study  entitled 
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Acquisition  Policy  Effectiveness:  Department  of  Defense 

Experience  in  the  1970s: 

.  .  .  Thus,  if  the  production  quantity  has  been 

reduced  since  DSARC  II  (a  common  occurrence),  an 
addit:  to  the  GE  is  required  to  bring  the 

pi  oar  cost  back  up  to  what  it  would  be  if  the 
orn  lly  programmed  quantity  were  to  be 
prO'  id;  if  the  production  quantity  has  been 
inc  c;sed,  a  reduction  of  the  CE  is  required. 

Thi  IS  accomplished  simply  by  deleting  the  cost 
ch..  ie  attributed  in  the  program's  SAR  to  quantity 
rnce.  (  35:80  ) 

This  me  iod  was  also  employed  by  the  Management  Consulting 
and  Resiarch,  Incorporated,  in  theiv  analysis  of  DoD  weapons 
system  cost  growth  (43:6). 


Non-SAR  Programs  and  SAR  Programs  Whose  DE  and  MFC 
Quantity  Differ.  For  the  non-SAR  programs  and  the  SAR 
programs  whose  DE  quantity  differed  from  the  MFC,  the 
quantity  ad3U3tment  was  made  by  computing  the  product 
improvem.ent  curve  and  first  unit  cost  (T  )  for  the  CE  using 
ICIiOT.  ICLOT  13  a  computerized  program  which  uses 
regression  analysis  to  determine  the  slope  and  T-  values  ro: 
either  a  unit  or  cumulative  average  curve  when  the  lot  size 
and  the  cost  per  lot  are  known  (20:7-33).  The  unit  curve 
formulation  was  used  in  this  research.  The  costs  per  lot 
are  the  recurring  costs  in  constant  year  dollars.  The  T- 
and  slope  generated,  were  then  used  to  calculate  the  cost 
which  would  have  been  incurred  if  the  quantities  outli.ned  in 
the  .MFC  were  still  in  effect.  These  cost  were  derived  using 
ICPRO,  a  computerized  program  which  "computes  pronected 
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values  under  the  unit  cost  improvement  curve  theory  given  a 


slope  and  T.  or  the  average  cost  of  a  lot,  and  its  first  and 
last  units"  (20:7-32).  The  adjusted  quantity  costs 
resulting  are  recurring  cost  in  constant  year  dollars.  The 
percentage  of  recurring  cost  to  total  cost  of  the  CE  was 
used  to  calculate  the  total  cost  of  the  adjusted  estimate. 
The  December  1989  OSD  Weighted  Inflation  Rates  (Appendix  C) 
were  used  to  convert  the  constant  year  to  then  year  dollars. 

Calculating  the  Cost.  Factor  (CF) 

To  determine  the  amount  of  cost  growth  the  adjusted 
current  estimate  (GE)  was  divided  by  the  MPG .  The  resultant 
number  is  the  cost  factor  (CF). 


where  (2 

d  =  development 
p  =  production 

A  CF  greater  than  one  implies  there  was  some  cost  growth  in 
the  program.  A  CF  of  less  than  one  i.-nplies  a  reduction  in 
cost  and  a  CF  of  unity  implies  there  was  no  cnange  in  the 
cost. 

Cost/Schedul e/Conf iqurat ion  Trends  Analysis 

Before  accomplishing  the  analysis  based  or.  the  three 
factors  contributing  to  cost  growth  identified  above,  the 
programs  were  analysed  to  determine  ir  any  trends  could  b'=‘ 
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found  based  on  the  dollar  value  of  a  program  and  its  cost 
factors  (CFs)  or  ECO  percentage.  This  analysis  was 
accomplished  by  graphically  plotting  the  CF  and  ECO  dollars 
against  cost  in  the  production  and  development.  Throughout 
this  research,  the  development  and  production  efforts  are 
treated  separately. 

Methodology  for  Data  Analysis 

The  four  step  process  below  was  used  to  analyze  the 
data  based  on  the  three  cost  factors  identified. 

Step  1,  First,  ail  the  observed  weapon  systems  were 
categorized  as  having  nigh  or  low  technical  risk  and  being 
of  stable  or  unstable  configuration.  The  data  on  the 
technical  risk  was  obtained  from  the  cost  documentation, 
discussion  with  program  managers,  and  cost  analysts  who 
worked  the  programs.  Systems  were  considered  to  be  of 
either  high  of  low  technical  risk.  T.hare  were  no  provisions 
made  for  medium  risk. 

Step  2.  The  technical  risk  was  then  compared  to  the 
configuration  stability.  The  configuration  stability 
a.ssessment  was  made  based  on  the  percentage  of  SCO  dollars 
to  the  most  probable  cost  estimate.  .According  to  The  AFSC 
Cost  Estimating  Handbook.  ECO  dollars  are  ".  .  .  widely 

accepted  throughout  the  Air  Force  as  representing  that 
amount  of  money  in  a  program  specifically  set  aside  for 
uncertainty"  (20:13-32).  The  higher  the  uncertainty  the 
higher  the  percentage  w.hich  must  be  set  aside.  Based  on 
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this  premise,  it  stands  to  reason  that  a  program  with  a 
higher  ECO  percentage  has  less  certainty  and  thus,  less 
configuration  stability.  The  data  base  was  divided  in  half, 
based  on  the  percentage  of  ECO  dollars  to  the  most  probable 
cost  estimate.  Placing  those  programs  with  higher  ECO 
dollar  percentages  in  the  low  stability  category  and  those 
with  the  lower  percentages  in  the  high  configuration 
stability  category.  The  four  possible  combinations  of 
technical  risk  and  configuration  stability  are  represented 
in  Table  3. 


Table  3 

Technical  Risk/Configuration  Stability 
Technical  Risk 
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Step  3.  The  next  step  in  the  analysis  was  to  add  the 
schedule  assessment.  Quadrants  I  through  IV  above  were 
further  broken  down  based  on  the  schedules  experienced  by 
the  programs  in  each  quadrant.  The  schedule  assessment  was 
analogous  to  the  convention  used  in  the  Independent  Schedule 
Assessment  Handbook  (49).  The  Handbook  defines  a  low  impact 
on  schedule  as  one  which  results  in  a  slip  of  less  than  one 
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month  (49:50).  A  medium  impact  is  one  which  results  in  a 
slip  of  at  least  one  month  but  less  than  three  (49:50),  and 
a  high  impact  is  one  which  results  in  a  slip  of  greater  than 
three  months  (49:50).  In  this  research  the  programs  were 
divided  into  two  categories,  low  schedule  impact  (Category 
A)  and  high  impact  (Category  B),  with  approximately  half  in 
each  category.  The  development  effort  schedule  assessment 
was  determined  based  on  completion  of  Development,  Test  and 
Evaluation  (DT&E) .  The  production  effort  schedule 
assessment  was  based  on  the  first  production  delivery 
schedul e . 

There  are  eight  possible  combinations  of  schedule, 
configuration,  and  technical  risk  which  are  represented  in 
Table  4  below.  All  eight  regions  of  Table  4  were  considerea 
to  be  pertinent  to  this  research,  due  to  the  many 
uncertainties  present  in  the  acquisition  environment. 


Table  4 

Technical  Risk,  Configuration  Stability,  and 
Schedule  Assessment 
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step  4.  Once  tha  systems  were  assigned  to  their 
respective  grids,  med: in  GFs  were  calculated  for  each 
category.  By  definition. 

The  median  of  a  set  of  observations  is  the  middle 
one  if  the  number  of  observations  is  odd  and  the 
average  of  the  middle  pair  if  their  number  is  even 
when  these  ohservations  are  arranged  in  increasing 
order.  (44:12) 

The  median  was  chosen  to  represent  the  estimated  potential 
cost  growth  becau.=  e  it  is  a  more  robust  measure  of  central 
tendency  than  the  mean  and  is  less  likely  to  be  unduly 
influenced  by  extreme  high  or  low  cost  factors.  According 
to  Newbold,  the  median  is  often  preferred  in  circumstances 
where  it  is  inappropriate  to  give  much  weight  to  extreme 
observations  (44:13). 

Range  of  Potential  Cost  Growth  Around  the  MPC 

The  last  step  in  the  analysis,  was  to  determi.ne  how  the 
information  obtained  above  could  b2  used  to  estimate  a  range 
of  potential  cost  growth  around  the  .MPC  in  answer  to 
investigative  question  :t3  In  the  following  paragraph  the 
method  for  obtai;iinc,  the  potential  cost  growth  range  is 
expl aineU . 

The  Median  CF  of  the  category  to  which  the  .MPC  is 
assigned  can  be  viewed  as  an  estimate  of  the  most  probable 
cost  growth  for  that  program.  T};e  upper  and  lower  bound  C  ; 
for  the  given  category  will  be  used  to  determine  -.ne  range 
of  cost  growth  around  thf*  most  probable  cost  estimate. 
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This  chiptisr  explained  the  methodology  used  to  answer 
the  'i.rivestic,at  1  ve  questions  posed  in  Chapter  I.  To  answer 
investigative  question  nl,  a  literature  search  was  conaicted 
in  Chapter  II  to  identify  the  factors  which  contribute  to 
cost  growth.  A  panel  of  experts  was  employed  to  supplement 
the  literature  search  and  ensure  the  factors  identirien  were 
applicable  in  today's  environment.  To  answer  investigative 
question  42,  a  method  of  accessing  a  program  based  on  the 
three  factors  identiiied  in  investigative  question  #1  was 
developed.  Lastly,  the  method  for  defining  a  range  of 
potential  cost  growth  around  the  most  probable  cost  estimate 
was  discussed.  The  methodology  set  forth  in  this  chapter 
will  be  conducted  and  analyzed  in  Chapter  IV  followed  by 
conclusions  and  recommendation  for  further  studies  in 


Chapter  V . 


Analysis  o£  Oata 
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chapter  Overview 

In  this  chapt  >r  three  factors  considered  to  be  major 
contributors  to  cost  growth  are  identified  by  a  panel  of  six 
cost  analyst  from  Aeronautical  Systems  Division,  and  applied 
to  a  data  base  consisting  of  sixteen  systems.  The  analysis 
is  followed  by  a  discussion  of  the  findings. 


Factors  Contributing  to  Cost  Growth 

In  step  one,  William  Kugel ,  Charlie  Clark,  Mike  Seibil, 
Dr.  Richard  Murphy,  and  the  author,  met  to  discuss  the 
reasons  for  cost  growth  contained  in  the  literature  and  to 
add  to  this  list  any  items  not  covered.  Once  the  group  fel*’ 
all  the  significant  sources  of  cost  growth  were  exhausted, 
the  following  list  of  factors  contributing  to  cost  growth 
was  derived: 

1.  Contractors  e.xperience  in  the  area  of  acquisition. 

2.  Contractor's  familiarity  with  the  Government's  way 


of  doing  business. 

3.  Risk  due  to  technical  advances. 

4.  Time  and  development  schedule  for  the  program. 

5 .  Degree  or  concurrence  between  engineering  and 
Manufacturing  Development  (2MD)  and  Production. 

6.  Whether  or  not  the  system  is  within  the  range  of 
historical  data. 

7.  System  requiremeiits/conf  igurat:  on  stability. 

3.  The  e;<i3tenca  ot  nonexistence  of  actual  cost  by 
functionality  and/or  work  breakdowa  structure  (W3S)  for 
analogous  systems. 
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9.  Schedule  slippage  or  variations. 

In  the  following  paragraphs  the  meaning  of  these  nine 
factors  as  discussed  by  the  cost  panel  will  be  explained. 

1.  Contractors  Experience  in  the  Area  of  Acquisition. 

Tills  factor  implies  that  if  contractors  had  experience 

in  the  area  of  acquisition  that  experience  can  be  applied  to 
the  new  system,  thus  reducing  risk. 

2.  Contractor's  Familiarity  with  the  Government's  Way 
of  Doing  Business. 

The  premise  underlying  this  factor  is,  if  contractors 
understood  the  peculiarities  of  doing  business  with  the 
government  the  cost  of  monitoring  the  contract  and 
rnrrfacti.ng  deficiencies  would  be  minimal. 

3.  Risk  Due  to  Technical  Advances. 

This  factor  considers  the  state-of-ti.  j-art  of  the 
technology  cf  the  system  being  produced.  The  relationship 
between  technical  advance  and  uncertainty  is  considered  to 
be  proportional;  the  more  technically  advanced  the  system, 
the  more  complex  and  the  greater  the  uncertainty. 

4.  Time  and  Development  Schedule  for  the  Program. 

Ti:  ;e  and  ueve i opment  schedule  in  this  context  refers  to 
the  amount  of  time  the  contractor  says  :t  will  take  to 
c  omp 1 e  t  e  the  project. 

5.  Degree  of  Concurrence  3etwe-Ti  SMD  and  Production. 

This  factor  attempts  to  capture  t.he  amount  c cost 

growth  due  so  the  .arnount  t  overlap  of  ie /e  1  opmen  s  and 
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production.  A  program  which  begins  production  before 
completion  of  development  tests  is  apt  to  experience  more 
cost  growth  than  one  that  begins  production  only  after  final 
testing . 

6.  Whetner  or  not  the  System  is  Within  the  Range  of 
Historical  Data. 

This  factor  is  of  concern  when  using  the  parametric  and 
analogous  estimating  technique.  Parametric  estimating,  as 
defined  in  Chapter  II,  often  depends  on  cost  estimati.ng 
relationships  and  the  analogous  method  looks  at  the  new 
system  as  compared  to  a  previous  system.  CERs  are  often 
calculated  usi.ng  regression  techniques.  There  are  some 
cautions  which  must  be  made  when  using  regression  analysis. 
Number  one,  as  the  item  being  estimated  moves  away  from  the 
center  of  the  data,  the  width  of  the  prediction  interval 
generally  increases.  Secondly,  if  the  system,  the 
independent  variaoie,  lies  far  beyond  the  range  of  past 
data , 

extreme  caution  should  he  exercised  since 
one  cannot  be  sure  that  the  regression  function 
which  fits  the  past  data  is  appropriate  over  the 
wiaer  range  of  f-he  independent  variable.  (  45:85  ) 

7.  System  Requirements/Configuration  Stability. 

In  the  development  arena  change  is  constant.  This 

factor  considers  cost  growth  that  is  the  result  of 
configuration  ciianges  . 

8.  ""he  Existence  or  Nouc  xistence  of  .^ctua’  Cost  by 
Functionality  and, •'or  WFS  for  .inalogous  Systems. 


If  the  analo^jous  systeir.  can  be  broken  down  by 
functionality  and/or  work  break  down  structure  with  .he 
benefit  jf  actual  costs,  the  resultant  cost  estimate  should 
be  able  to  predict  cost  more  accurately  than  if  this  were 
not  the  case.  A  major  assis'nption  is  that  the  inalogous 
system  was  correctly  identified. 

■5.  Schedule  Slippage  or  Variation. 

Unlike  number  three  above,  which  refers  to  t.he  number 
of  development  years,  this  factor  refers  to  cost  growth  due 
to  schedule  stretch-outs  or  compressions. 

Three  Maior  Contributors  to  Cost  Growth.  As  discussed 
in  Chapter  III,  after  identifyinj  the  factors  the  next  step 
was  to  have  the  members  of  the  panel  rank  the  factors  from 
one  to  nine  allowing  the  top  three  factors  to  share  the  rank 
of  one.  The  result.-  of  the  ranking  process  are  contained  in 
Table  5.  ih‘=*  "^umbers  far  left  column  represent  the  .nine 

cost  growth  factors  (cJFs)  identified  above.  The  column 
headings  are  the  rankings  from  1  through  9.  The  numbers  in 
the  body  of  the  table  represent  how  many  of  the  six  panel 
..'.s  oeis  ranked  the  identified  factor  at  a  given  level.  The 
results  of  the  ranking  clearly  indicates  that  configuration 
stabil:,.ty  (#7;  is  thought  to  be  a  ma^or  factor  influencing 
cost  growth.  Six  of  tiie  six  pan  •!  members  ranked  this 
factor  .number  one.  Fout  of  the  six  cost  analysts  ranked 
risk  due  to  t^chniial  advances  3 ;  as  number  one.  The 
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Table  5 


Ranking  of  Factors  Contributing 
to  Cost  Growth 
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uisCiiri^c^^  tTXS/^  3^s  nti*rt4rfu!r  scvcrx*  T*^40 
analysts  who  ranked  risk  due  to  technical  advances  as  number 
seven,  acknowl edced  that  technical  risk  is  an  important 
consideration  when  discussing  cost  growth;  however,  they 
felt  that  with  t  .le  current  state  of  technological  maturity, 
technical  risk  is  not  a  maior  factor  in  today's  environment. 
The  third  factor  receiving  high  ranks  from  the  cost  panel 
was  schedule  slippage  or  variations.  Fifty  percent  of  the 
panel  members  felt  this  was  a  ma]or  factor  and  one  panel 
member  ranked  it  fourth.  Though  there  were  some  disparities 
in  the  ranking;  configuration  stability,  technical  risk,  and 
schedule  variations  were  the  three  cost  factors  determined 
by  the  panel  cost  analyst.^  to  have  the  most  significant 
impact  on  cost  growth  in  today’s  environmenc. 
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Data  Collection 

Currently,  a  central  data  base  does  not  exist  which 
lists  all  the  weapons  systems  procured  at  Aeronautical 
Systems  Division  over  the  past  decade.  To  assist  in 
identifying  the  applicable  system,  William  Kugel ,  and  the 
Research  and  Cost  Division,  Directorate  of  Cost,  provided 
the  initial  list  of  systems  procured  in  the  last  decade. 

The  resultant  list  consisted  of  thirty  programs.  The  list 
was  reviewed  to  ensure  all  systems  had  completed  at  least 
three  years  of  development  and  that  all  programs  were 
actually  procured  in  the  1980s.  Those  systems  which  did  not 
fit  the  three  year  development  criteria  were  immediately 
discarded  from  the  list.  Additional  systems  were  deleted 
due  to  unavailability  of  data.  The  final  list  of  programs 
used  in  this  research  is  contained  in  Table  6.  .Appendi.x  .A 
contains  a  description  of  the  programs. 


Table  6 

Programs  Used  in  the  Research 


AC-130  Gunship 
Advance  Cruise  Missile 
Air  Force  One 
Advance  Tactical  .Air 

Reconnaissance  System 
3 - 1 B  Si mu 1  a t o r 
0-17  (C-Xl 

:-17  Aircrew  Training 
System 

C-17  Maintenance 
Traini.ng  Device 


C-130 

it  Talon 

.-se  .Missile  .Mission 
Control  Aircraft 
F-111  Digital  Flight 
Control  Syst em 
LANTIRN 

LANTIRN  SIMULATOR 
SR.AM  I  I 
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Tables  7  and  8,  contain  the  data  collected  on  the 
selected  programs  for  the  development  and  the  production 
effort  respectively.  The  MFC  estimates  in  the  tables  were 
obtained  from  Source  Selection  cost  documentation  found  in 
ASD ' s  Cost  Library.  The  current  estimates  were  taken  from 
the  latest  system  program  office  approved  cost  estimates  and 
from  the  December  1990  Selected  Acquisition  Reports  ( SARs ) 
and  adjusted  for  quantity  as  needed. 


Table  7 

Development  Effort  ($  in  Millions) 


SYSTEM 

MFC  $ 

CE  $ 

CF 

SCHED 

VAR 

TECH 

RISK 

! 

SCO  1 
%  1 

a 

26Q . 50 

265.03 

1 . 02 

12 

mns 

H 

0 .01  1 

ACM 

1492 . 40 

1755 . 20 

1.13 

4 

mo.s 

H 

0.04 

AF-1 

48 . 90 

40 . 40 

0.33 

2 

yrs 

L 

0.01 

ATARS 

179 . 83 

237 . 30 

1.60 

22 

mo3 

H 

0 .08  i 

31B  SIM 

142 . 87 

n 

mm 

L 

0.30  i 

C-17 

- 1 

3567 . 42 

5595 . 30 

1  .  57 

15 

mos 

L 

0.03  1 

C-i7  ATS 

57 . 49 

75  .  43 

n 

6 

mos 

IMI 

BBi 

C-17  MTD 

54 . 89 

93 . 44 

1.70 

11 

mos 

L 

0.36! 

C130  ATS 

12 . 6 

13.03 

1.03 

0 

mos 

L 

0,12  i 

COMBAT  T 

46.3 

104 . 30 

2 . 25 

19 

mos 

0 .14 

CMMCA 

54 . 50 

- - - 

53.04 

1.06 

15 

mos 

H 

0.03  ; 

FlllDFCS 

59.50 

63 . 92 

1.07 

1 

MOS 

0 . 40 

lantirn 

548 . 90 

601 . 30 

1.10 

17 

mos 

H 

0.03 

LANTIRN 

SIM 

29.07 

28 . 60 

0 . 98 

17 

mos 

r 

u 

0.02 

SRAM  II 

991 . 94 

1 . 28 

2 

yrs 

H 

mm 
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Of  the  sixteen  systems  reviewed,  fifteen  are 
represented  in  the  development  effort,  because  the  G-20 
Aircraft  Program  is  a  production  only  program.  The  cost 
factor  for  Combat  Talon  is  noticeably  higher  than  the  other 
programs  because  the  program  was  initiated  as  a  production 
only  effort.  Some  development  money  was  initially  included 
in  the  MFC  estimate  to  allow  for  expected  redesign  work  in 
the  production  effort.  When  it  became  apparent  that  a  more 
extensive  development  program  was  necessary,  a  full  scale 
development  effort  was  added  to  the  program's  baseline. 
Though  this  does  represent  an  anomaly,  the  program  was  not 
discarded  from  the  data  base,  because  the  intent  of  the 
study  is  to  capture  all  cost  growth,  including  anomalies. 

Fourteen  systems  are  represented  in  the  production 
effort.  The  Cruise  Missile  Mission  Control  Aircraft  does 
not  include  a  production  effort.  The  Advanced  Tactical  Air 
Reconnaissance  System  (ATARS)  production  effort  was  not 
included  due  to  complications  in  the  current  estimate,  which 
are  currently  being  reworked. 

The  Advanced  Cruise  Missile  (ACM),  C-17  aircraft, 
LANTIRN,  and  SRAM  II  programs  all  experienced  a  quantity 
’■eduction  from  the  MPC  estimate  to  the  production  estimate. 
All  of  these  programs  are  Selected  Acquisition  Report  (SAR) 
programs  ^nd  all,  except  the  C-17,  have  the  same  production 
quantities  in  the  development  estimate  baseline  reported  in 
the  SAR  and  the  MPC  estimate.  In  these  cases,  the  current 


57 


estimates  were  adjusted  back  up  to  the  original  quantity  by 
adding  the  decrease  due  to  quantity  change  reported  in  the 
3AR  to  the  current  estimate.  This  method  could  not  be  used 
for  the  C-17  program,  because  the  MPC  estimate  differed  from 
the  development  estimate.  The  development  estimate  (DE)  as 
defined  in  Chapter  II,  is  the  baseline  to  which  many 
programs  are  compared,  normally  developed  late  in  the 
Demonstration  and  Validation  Phase.  The  MPC  in  the  G-I7 
program  represents  a  quantity  of  132  units,  while  the  DE 
represents  a  quantity  of  210  units.  The  latest  SAR  reports 
a  quantity  reduction  of  90  units  and  an  associated  cost 
reduction  of  $11039  million.  If  the  current  estimate  of 
$29351.9  million  were  adjusted  for  quantity  by  .simply  adding 
back  in  the  reduction  associated  with  the  quantity  change, 
the  adjusted  current  estimate  would  be  $40390.9  million. 
However,  if  t.his  current  estimate  were  compared  to  the  MPC 
to  determine  the  ^  jst  growth  the  amount  would  be 
artificially  high,  because  the  MPC  represents  132  units 
versus  210.  The  current  estimate  was  normalised  ro  reflect 
a  quantity  of  132  production  units.  This  normalisation  was 
accomplished  by  first  calculating  a  cost  improvement  curve 
for  the  current  estimate  quantities,  using  ICLOT  to  develop 
the  theoretical  first  unit  cost  i.T  '  and  slope.  Cnee  the 
slope  and  T-  were  calculated  ICPRO  was  used  to  estimate  the 
cost  for  132  units  using  the  same  spend  profile  as  the  MPC 
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Fi'^ure  3.  Development  Coat  Factor/Cost  Comparison 


Figure  4.  Development  ECO/Cost  Comparison 


From  Figures  3  and  5,  there  does  not  appear  to  be  a 
relationship  between  the  size  of  a  program  and  the  cost 
factor  in  either  the  development  or  the  production  efforts. 
However,  a  relationship  does  appear  to  exist  between  the 
total  cost  of  the  program  and  the  ECO  percentage.  This 
trend  is  not  as  apparent  in  the  development  effort  as  it  is 
in  the  production  effort;  however,  it  appears  that  the 
larger  programs  tend  to  have  a  smaller  percentage  of 
engineering  change  orders  (Figures  4  and  6).  Despite  this 
trend,  the  ECO  percentages  were  considered  to  be  good 
indicators  of  configuration  stability,  based  on  the 
assTimption  that  ECOs  represent  the  uncertainty  in  an 
estimate  (20;13“32)  and  that  the  greater  the  ujiuet  laiut y  the 
greater  the  percentage  of  ECO  required  to  cover  changes  and 
rework  (20 : 13-36) . 

Data  Analysis 

The  first  step  in  the  data  analysis,  as  described  in 
Chapter  III,  was  to  categorize  the  data  ba.sed  on  technical 
risk  and  configuration  stability.  Tables  9  and  11  depict 
the  technical  risk  and  configuratio  stability  of  the 
development  and  production  efforts  respectively. 

Observations  from  Table  9  revealed  that  the  h-igh 
technical  risk/low  configuration  stability  category  '■ended 
to  have  the  highest  cost  factors,  while  the  low  technical 
risk/high  configuration  stability  category  has  lower  c  j.-3t 


factors  . 


Table  9 


Technical  Risk/Configurati on  Stability 
Development  Effort 


Technical  Risk 


High 


Low 


(0 

4J 

U1 

c 

o 


u 

3 

O' 


a 

0 

'■) 


j: 

O' 


3 

o 

.J 


System 

CF 

System 

CF 

AC-i30  Gunship 

1.02 

C-17 

1.57 

ACM 

1 . 18 

CMMCA 

1.06 

LAWTIRN 

i.iO 

LANTIRN  Sim 

.  98 

C-17  ATS 

1.31 

AF-1 

.  83 

System 

QL 

System 

CF 

SRAM  II 

1 .  28 

B - IB  Sim 

1.31 

ATARS 

1 .  60 

F-111  DFCS 

1.07 

Combat  Talon 

2  .  25 

C-17  MTD 

1.70 

C-130  ATS 

1.03 

appeared  that  the 

high 

technical  risk 

program. 

higner  CFs  tha:i  the  low  technical  risk  programs.  To  test 
this  a  ccmparison  was  made  between  the  two  categories.  The 
median  GF  for  the  high  technical  risk  programs  was  1.23 
while  the  median  CF  for  the  low  techiiical  risk  programs  was 
1.07;  consistent  with  expectations.  A  similar  comparison 
was  made  between  low  and  high  conf igurat : on  stability.  The 
median  CF  was  1.08  for  programs  wich  high  configuration 
stability  and  1.31  for  those  programs  with  low  configuration 
stability.  Once  again  the  re.sults  were  cons  1 1-; ent  with 
expectation.') . 

A  I'jok  at  the  .median  cost  factor':  (Tablt-  10)  for  each 
quadrtnt  of  Table  9  reinforces  the  i.^'ends  -ohserved  above. 


Table  10 

Technical  Risk/Configuration  Stability 
Median  Development  Cost  Factors 


Techi.ical  Risk/ 
Config.  Stability 

Median 

CF 

High/High 

1.10 

Low/High 

1.06 

High/ Low 

1.60 

Low/ Low 

1.19  1 

The  median  CF  for  Quadrant  I,  high  technical  risk/high 
configuration  stability  is  1.10;  Quadrant  II,  low  technical 
risk/high  configuration  stability  is  1.06;  Quadrant  III, 
high  technical  risk/low  configuration  stability  is  1.60;  and 
quadrant  IV,  low  technical  risk/low  configuration  stability 
IS  1.19.  From  these  observations,  configuration  stability 
appears  to  have  a  greater  influence  on  cost  growth  than 
technical  risk.  In  both  cases  where  configuration  stability 
13  unfavorable  (low),  the  highest  median  cost  factors  are 
chf.erved;  1.60  and  1.19.  On  the  other  hand,  the  median  CF 
13  relatively  low  in  one  instance  where  the  technical  risk 
IS  high. 

■"'•veral  observations  can  be  made  from  a  precursory  look 
at  Table  11.  One  of  the  most  notable  is  that  the  low 
technical  ri3k,^high  configuration  stability  category  h'S  the 
lowest  cost  factors;  0.77,  0.99,  and  1.03.  Thi-S  category 
also  contains  one  of  the  highest  cost  factors  in  the  dat.i 
base  . 
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Configuration  Stability 


Table  11 


Technical  Risk/Configuration  Stability 
Production  Effort 

Technical  Risk 

High  Low 


c> 

•H 

X 


3 

O 

.J 


Another  observation  is  that  the  high  technical  risk 
category  overall,  appears  to  have  lower  cost  factors  than 
the  low  tecitnical  risk  categories.  The  median  cost  factor 
of  the  high  t&c.hnical  risk  category  i.s  1.14  compared  to  1.34 
for  the  low  technical  risk  category.  There  may  be  several 
reasons  why  the  nigher  risk  program.*;  experience  less  cost 
growth  than  the  Lower  risk  programs  in  the  production 
eftoit.  Cr.e  of  which  "ould  be  extra  attention  placed  on 
proper  design  of  programs  thought  to  be  of  higher  risk.  T'-.e 
'"\edian  cost  factors  of  the  high  and  low  configuration 
stability  categories  are  1.12  and  1.29  respectively' 
indicating  that  pregrams  with  high  configuration  stability 
tend  to  experience  less  c:st  growth  than  those  with  low 


System 

CF 

System 

QL 

Combat  Talon 

1.29 

C-17 

1.83 

ACM 

1.14 

C-20 

.  77 

LAffTIRN 

1.09 

LANTIRN  Sim 

1.59 

C-i7  ATS 

.  99 

AF-1 

1.03 

System 

QL 

System 

CF 

SRAM  II 

1.23 

B-lB  Sim 

1.96 

F-111  DFCS 

1.34 

AC- 130  Guns hip 

1.12 

C-17  MTD 

1.64 

C-130  ATS 

1.09 
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configuration  stability.  Table  12  depicts  this  trend.  The 
programs  with  low  configuration  stability  .nave  the  hxghest 
median  cost  factors.  This  suggests  that  configuration 
stability  13  perhaps  a  more  significant  contributor  to 
cost  growth  in  the  production  effort.  In  the  development 
effort,  the  smallest  median  cost  factor  was  reported  in  the 
low  technical  risk/high  configuration  stabil^-ty  category. 

Table  12 

Technical  Risk/Configuration  Stability 
Median  Production  Cost  Factors 


Technical  Risk/ 
Config.  Stability 

.Median 

CF 

High/High 

1 

1.14  ! 

uOw/High 

I-* 

o 

High/ Low 

00 

» — 1 

Low/ Law 

n 

1.49  i 

One  of  the  ma^or  differe.nces  in  the  development  and 
production  estimates  is  in  the  category  which  has  the 
highest  cost  factor.  In  tie  development  estimate,  the  high 
technical  risk/ low  configuration  stability  category  had  the 
highest  median  CF ,  ct  1.60.  In.  tne  production  effort  low 
technical  risk/ low  configuration  stability  had  the  highest 
median  CF  of  1.4'3.  From  these  observations  it  appears  that 
high  technical  risk  is  more  important  to  cost  growth  during 
the  development  effort;  than  in  the  production  phase.  The 
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next  step  in  the  analysis  Jas  to  incorporate  the  schedule 
factor  into  the  ecuatron. 

Based  on  the  schedule  variations  contained  in  Tables  7 
and  8,  low  schedule  assessment  was  defined  as  one  which 
je  ults  in  a  slip  of  less  thE  n  si.-w  months  for  the  purpose  of 
this  study.  Tables  13  anu  14,  refer  to  those  programs  that 
are  no  more  than  six  months  behind  schedule  as  Category  A, 
and  those  tha -  are  more  than  six  months  behind  schedule  as 
Category  3.  The  quadrants  I  through  IV  represent  the  four 
scenarios  from  Tables  9  and  10.  Quadrant  I  represents  high 
technical  risk/high  configuration  stability,  Quadrant  II 
represents  those  with  low  technical  risk/high  configuration 
stability"  Quadrant  Tt^,  high  ’■echnioal  ri.sk  / low 
configuration  stability;  and  Quadrant  IV,  low  technical 
risk/low  configuration  stability. 

Of  the  fifteen  development  systems,  only  four  were  less 
':han  six  months  behind  schedule.  Seventy- four  percent  of 
the  programs  were  greater  than  six  months  behind  schedule  in 
tne  development  effort  and  sixty-four  percent 
i'nine  out  of  fourteen)  of  the  prcduct.ion  effort  were  either 
greater  .han  or  proiected  to  be  greater  than  six  montns 
behind  schedule.  The  high  technical  risk/ low  configuration 
staoi  1 1  ty/ 1  ow  schedule  impact  categor'/  w;‘3  not  representea 
in  •:  oe'  ,  ’ opment  nor  the  production  ef torts. 

■  .e--  ;■  ‘  -e  ca’culated  for  each  cell 


I  and  are  pr-^sented  in  Tables 
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15  and  16  based  on  the  technical,  conf icuration,  aid 
schedule  assessments.  The  upper  bound  CPs  in  the  tables 
represent  the  highest  CF  contained  in  a  given  category.  The 
lower  bound  CFs  are  simply  the  lowest  CF  in  the  category. 

In  the  three  cells  containing  only  one  observation,  the 
observation  is  recorded  in  the  table  as  the  median. 


Table  15 

Development  Potential  Cost  Growth  Range 


lech 

Risk 

Conf ig 
Stabi 1 ity 

Schedul e 
Impact 

Upper 

CF 

Med 

CF 

Lower 

CP 

High 

High 

Low 

1 . 18 

High 

High 

high 

1.10 

1.06 

1.02 

Low 

High 

Low 

1.31 

Low 

High 

High 

1.57 

1.02 

.83 

High 

Lew 

High 

2.25 

1 . 60 

1.28 

Low 

Low 

Low 

L_.^  •  9  J 

1.05 

1.03 

Low 

Low 

High 

1.70 

1 .51 

1  .  31 

The  lowest  median  CF  in  the  development  etiort  was  1.02 
in  the  low  technical  risk/high  configuration  stabi 1 ity/high 
schedule  impact  category.  The  category  with  the  highest 
median  (_F  is  the  high  technical  risk/low  configuration 
stability/high  schedule  impact  category,  at  1.60.  The  two 
I'.ighest  median  CFs  reported  in  Table  15  both  have  high 
schedule  risk  and  low  configuration  stability.  Of  the  four 
categories  containing  high  schedule  impact,  two  have  the 
highest  median  CFs,  1.60  and  1.51,  while  the  other  two  have 
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two  of  the  lower  CFs ,  1.02  and  1.06.  In  the  in.stancea  where 
a  high  schedule  risk  is  associated  with  a  low  median  cost 
factor,  it  is  coupled  with  high  configuration  stability. 

When  coupled  with  low  configuration  stability,  the  median  CF 
tended  to  be  high.  There  was  no  apparent  relationship 
between  technical  risk  and  the  schedule  impact.  One 
observation  which  can  be  noted;  however,  is  that  of  the 
three  factors  configuration  stability  appears  to  have  the 
most  influence  over  cost  growth.  It  appears  that  if  either 
an  unfavorable  schedule  or  technical  risk  assessment  is 
combined  with  an  unfavorable  configuration  assessment,  the 
median  cost  factor  will  be  high.  The  converse  does  not 
appear  to  be  true.  If  the  configuration  stability 
assessment  is  high  (favorable),  combining  it  with  a  program 
with  low  technical  risk  or  a  low  schedule  risk  will  not 
guarantee  a  low  cost  factor.  In  fact  with  al factors 
reporting  favorably,  the  cost  growth  factor  in  scenario 
three  (low  technical  risk,  high  configuration  stability,  low 
schedule  risk)  was  1.31,  one  of  the  highest  reported  in  the 
tabl e . 

From  Table  16,  the  highest  median  CF  reported  in  the 
production  effort  was  1.30,  which  occurred  in  the  scenario 
where  the  schedule  and  configuration  assessments  were 
unfavorably.  Schedule  risk  and  configuration  stability  do 
not  appear  to  have  the  same  correlation  witnessed  in  the 
development  effort.  The  secc  id  highest  median  CF  in  the 
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production  effort  is  the  same  scenario  which  produced  the 
lowest  median  CF  in  the  development  estimate,  in  which 
technical  risk  and  configuration  were  reported  favorably, 
while  the  schedule  was  unfavorable.  In  all  instances  where 
the  schedule  risk  was  high  the  median  CF  v/as  at  least  1.13. 
The  median  cost  factor  overall  seems  to  be  higher  in  the 
production  effort. 


Table  16 


Production  Potential  Cost  Growth  Ra.nge 


Tech 

Conf  ig 

Schedul e 

Upper 

.Med 

Lower 

Risk 

Stability 

Impact 

CF 

CF 

CF 

High 

High 

Low 

1.14 

High 

High 

High 

1.29 

1.19 

1.09 

Low 

High 

Low 

.99 

.88 

.  77 

Low 

High 

High 

1.83 

WB 

.  83 

High 

Low 

High 

1.23 

1.18 

1 . 12 

Low 

Low 

Low 

1.34 

■— 

1 .09 

Low 

Low 

High 

1.96 

1 . 30 

■W 

Calculating  the  Range  of  Cost  Growth 

The  first  step  in  determining  the  range  of  potential 
cost  growth  is  the  completion  of  the  MPC  estimates  by  the 
source  selection  cost  panel .  Once  the  estimates  are 
^mplete,  the  cost  analyst  assigns  each  MPC  estimate  to  one 
of  the  categories  in  Table  4  based  on  the  technical, 
schedule,  and  configuration  risk  assessments  used  by  the 
analyst  while  developing  the  estimate.  The  median  CF ,  upper 
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and  lower  bound  GF  from  Table  15  and  16  can  be  i,‘.v..\ed  to  ■;  iin 
a  feel  for  the  range  of  coat  growth,  exipeciencad  In  recent 
programs . 

a  Uliana  rv 

In  this  chapter  nine  factors  contributing  to  cost 
growth  were  analysed  and  three  were  chcsen  by  a  panel  of. 
experts  to  be  the  r'ost  significant;  technical  risk,  schedule 
risk,  and  configuration  stability.  The  most  probable  cost 
estimate,  current  estimate,  schedule  variation,  and  the 
percentage  of  en.gineecing  change  order  costs  to  the  most 
probable  cost  estimate  -rere  collected  from  a  data  base  of 
sixteen  systems  procured  in  the  last  decade.  The  cost 
gruwlii  (jf  the  sy~^terris  was  analyzed  based  on  the  three 
factors  above  to  determine  f  potential  cost  growth  can  be 
forecasted  based  on  an  assessment  of  the  three  factors.  The 
conclusions  to  his  analysis  and  recommendation  for  further 
study  are  contained  in  the  next  chapter. 


V.  Conclusions  and  Recominendati ons 

Chapter  Overview 

The  objective  of  this  research  was  to  develop  a  method 
of  predicting  a  range  of  potential  cost  growth  around  the 
moat  probable  cost  estimate.  The  author  sought  to  develop 
thir.  method  by  researching  the  cost  growth  experienced  in 
recent  programs  and  categorizing  these  programs  based  on 
several  factors.  These  factors  were  derived  from  a 
literavvire  search  and  ultimately  narrov;ed  down  to  three 
factors  by  a  panel  of  cost  analysts.  In  the  chapter  the 
?:e.sults  of  this  research  will  be  discussed  by  answering  the 
investigative  tiUesrions  presented  in  Chapter  I.  This 
discussion  will  be  followed  by  recommar dati ons  for  further 
study . 

Trivestioative  Oue stion  #1 

Invest.! qative  question  #1  seeks  to  determine  the 
sources  of  cost  growth  in  the  acquisition  of  weapon  systems. 
The  answer  to  thi.'i  questica  was  pursued  using  a  two  step 
process.  The  first  step  involved  a  literature  searci.  to 
ascertain  what  faccors  are  considered  to  be  jor  driver.s  of 
cost  growth.  Cue  of  the  conclusion.s  obtaineu  from  the 
literature  search  is  that  the  factors  cant  ri  but  i?-' j  to  cos.: 
growth  are  not  cQnst..int,  but  cha.nqe  over  time.  The  dynamic 
envir  onment  emphasized  the  need  to  unaei  ?.  t 
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he  factors 


affecting  cost  growth  in  the  current  environment  when  trying 
to  control  cost  growth. 

In  the  second  step  a  list  of  factors  contributing  to 
cost  growth  was  generated  through  the  combined  efforts  of 
the  literature  and  working  group  of  five  cost  ■'nalysts. 

T'  is  list  of  factors  was  individually  ranked  by  a  panel  of 
cost  analysts  to  determine  the  three  factors  which  are 
considered  to  be  the  most  significant  drivers  of  cost  grovjth 
in  today’s  environment.  The  three  items  identified  were 
technical  risk,  configuration  stability,  and  schedule  risk. 
Once  these  factors  were  identified  the  next  step  was  to 
determi.ne  licw  this  information  could  be  used  to  help 
estimate  potential  cost  growth;  the  question  asked  in 
investigative  question  <^2. 

Investigative  Question  #2 

In  order  to  answer  investigative  question  ?2  sixteen 
programs  were  analyzed  to  deiecmine  if  there  were  any 
correlations  between  the  cost  growth  experienced  in  the 
program  and  the  three  factors  determined  to  be  ma^or  cost 
drivers.  Several  implications  can  be  made  from  the  data. 

In  the  following  paragraphs  they  will  be  explained  as  it 
relates  to  the  development  and  the  production  efforts. 

Ooservations  :.n  the  Development  Effort  .  Wne  n  the 
effect  of  all  three  factcr.s  are  cunsiaerod  together  in  the 
development  effc't,  configuration  stability  tends  to  have 

"  t; 
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more  influence  on  cost  growth  than  the  other  fc>ctot3.  The 
analysis  suggest  that  significant  cost  growth  should  be 
expected  if  the  program  is  operating  in  an  environment  with 
low  configuration  stability  and  high  schedule  risk.  Though 
high  configuration  stability  does  not  guarantee  minimal  cost 
growth,  the  cost  growth  experienced  in  these  prcigrams  tend 
to  be  less  on  average  than  those  with  low  configuration 
stabi 1 ity . 

T.t  also  appears  that  reducing  technical  risk  will  not 
significantly  decrease  cost  growth  if  there  is  a  high 
probability  that  the  schedule  will  slip  si<  months  or  more 
However,  to  minimize  potential  cost  growth  the  configuration 
must  be  stabilized  ns  early  as  practicable  .in  the 
development  effort  since  the  programs  with  h.ighec 
configuration  stability  tended  to  experience  less  cost 
growth  on  average,  regardless  of  the  behavior  of  technical 
risk  and  the  schedule  assessment. 

Observations  in  the  Production  of  fort.  When  all  thre.e 
factors  are  considered  together  conf .iguration  stability  does 
not  appea;  to  be  as  inf  1  uenTiiai  in  the  production  eifurt 
it  did  in  the  development  effor'^.  Schedule  risk,  however 
appeared  tu  be  *he  ;i-ost  influential  cost  driver.  In  ;■  1  1 
instances  where  the  schedule  ri.S'C  was  hign.  t.he  Go.=:t  growth 
exceeded  eighteen  percent.  Wheri  configuration  stability  and 
techriioul  risk  were  viewed  apart  from  schedule  mk: 
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however,  con£igura':ion  stability  was  a  major  1: actor 
influfiiicing  cost  growth. 

This  suggests  that  there  is  a  stronger  relationship 
between  conf igurfition  .-’tability  and  schedule  in  the 
production  effort.  Therefore,  in  order  to  minimize  cost 
growth  in  the  production  effort  the  conf iguration  must  be 
stable  and  the  schedule  must  be  realistic.  Tecnnical  risk, 
is  not  a.e  influential  in  the  production  effort  as 
configuration  stability  and  schedule. 

The  next  step  in  the  analysis  was  to  determine  how  this 
-.nformation  can  be  utilized  by  cost  analysts  to  help  control 
cost  grcwth.  This  issue  will  be  discufj.sed  in  the  following 

r*  ^  #T  V  ^  n  V. 
r  *-  . 

Irvestigat:  v e  Question  .3 

This  investigative  qua.';  ion  sought  to  determine  how  the 
inf Qrri;at',on  used  above  could  be  used  to  help  estimate 
potential  cost  growth  around  the  most  probable  cest 
esti.mat.'.  The  cost  anaiyst  car  use  the  information  gained 
in  this  research  to  gain  a  feel  for  tht  range  of  cost  growth 
exper:  sneed  in  similar  programs,  by  observing  the  cost; 
grerwth  histories  in  Tables  1;.  and  16  in  Chapter  VI. 

Due  to  the  I  im.  ted  data  set,  no  statistical,  inferences  can 
be  oc'-.'ined  from  the  data.  t’lth  a  larger  data  set  a.":  i  a 
signi  t :.  car  t  number  of  observations  in  each  C‘:I;  ‘-.oe  me  t. r, c  .i 

could  pi.Qvic'e  some  stat  i  stic-il  interval",  tar  pov.ci.itia  -.'S' 


R«.cc:'Amndat.ons  ‘cql  Furtbcn*  Research 

11.  i;3  otrongly  reconin'.ended  ^,hat  rnore  research  he 
acccmp  1 :. GT; -.'d  r*".  this  area.  Though  the  literature  suggest.^ 
tnat  there  inaioL  ditterences  in  the  p.'- ocurement 

en  vi  r  Gnrv'.r,  t  ■■.etwien  periods,  thn  applicable  data  set  can  be 
signi i icant  1  y  .'.-icreased  by  norir.al i sing  tor  these 
di f  f  arevices  .  .NoriTia  1  i sing  for  the  d:- ff  erence.s  ,  such  as 
longer  dev  o  1  opr.i-cnt  cycles  in  the  eighties,  can  also 
elirninate  the  ’.'rc.oleni  of  deali.ng  wit.h  such  current  data. 

This  re.search  tvc-.sed  ori  programs  that  •■.ere  initiated  in  the 
last  docavle  nave  been  in  development  or  pr  eduction  for  at 
least  three  ye.ar;:' .  hr.e  of  the  drawbacks  of  u;-iing  s’jch 
c\irrent  data  is  t;/at  -t'titinates  had  to  be  used  instead  of 
actuals.  Tho'  gh  ico'iv:  '.t  these  rrograna  are  mature,  there 
are  many  factors  wh'd.-.-.h  c in  change  t.ae  current  cost  growth 
i rends.  Jusc  is  Svirrn-  -.he  prograrr-s  are  mature,  there  are 
.some  ..ihich  -'-re  iclativ  /  ’otnj.;  and  may  not  have 
experienced  the  most  .s  i  gu  v  r  ..  ■.ru'.  •;cst  growth.  Ideally  it 
'j".uld  better  to  de-.  1  -.Jit"-  -  u., 'o.  because  .vn  dealing  ’with 

actual  costs  over  tho  ;;  r  ■u;j  r  i;r.  you  do  not  run  into  t.he 

'■roo'em  or  ■'Oinpari.ng  .^y  .-.t e;r-..  differeni  m-.itui;ity  levels. 

:  'tu:  •>  re,  earch  vioul'.l  htT.eti  1  v  j.t  ubcer'/'' ug  the  cor  t  growth 
vior  of  programs  witii  '  .•vats  a’/ailable. 

.oacther  issue  r.iin.'id  'du.'.'.:;,  '.he  re^'eaiu^h  was  t.he 
'ipr-4  rent  re  1  -itioiuoViip  V'et-Joe.-i  icil-ir  value  or  the 

proq'.'dn  and  the  number  of  -riig ...i ■.•  •  ■  •'jhange  orders  (,  rCOs )  . 
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It  se^^aed  that  the  smaller  program:-  had  &  higher  percentage 
of  ECOs .  Suggesting  thar  future  studies  should  segregate 
progr/ams  by  sise. 

The  number  of  programs  greater  than  six  months  behind 
schedule  indicates  that  there  is  a  dire  need  for  hetcer 
schedvle  assessments.  An  analysis  of  the  various  reasons 
for  schedule  slippage  can  provide  in.sigiit  into  the  areas 
wbsre  emphasis  musr  b<“  placed  to  improve  the  overall 
sohedu.'  e  performance, 

The  hPC  estimates  can  usually  be  classified  as  Category 
IV  no  uctuais  available,  system  description  is  fair  to 
gccd,  some  data  available  for  analogies,  factors, 

■••jaTamet  »-ics  .  etc.,  or  Category  V  --  uncertain  system 
descr.ption  and  de.ta  available  for  analogies,  factors 
par  smet' rics ,  etc.  are  .sparse  or  hove  limited  value  to  sysrem 
being  esr.iwia '■  ed .  the  r  ange  of  co.st  growth  around  the  MFC 
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of  the  five  categories  -if  confidence  deSiaed  in  .AI'SCR  .I”!  t 
and  in  Chapter  III  .of  this  research. 

Sumnva.r  V 

This  :  esoare.h  sought  to  determine  a  meched  *"0  es  i  imn  t 
tne  po.:ential  cusr  crowih  arciiiid  'he  most  pr.ioable  -.0.01 
estimate  basec.  on  the  behavior  of  t.hree  ma;;o’'  cos:  dri  /trs; 


and  schedule  risk. 


technical  risk,  configuration  stabilit-/. 

The  data  obtain'=*d  in  this  research  is  useful  in  providing 
cost  analyst  with  a  rough  order  of  magnitude  of  what  can  be 
expected . 
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Appendix  A:  :?roqraiTi  Des cr ,  ?r  i ons 

AC-130^^  Gunship  (2:2-3) 

The  AC-130U  Gunship  program  will  purchase  rwelve  C-*30H 
airframes  which  will  be  converted  tc  side-firing  gunsr.ips. 
The  gunship  will  incorporate  improvtd  night/adverse  weather 
capability  with  improved  survivability,  precision  fire 
support,  and  worldwide  deployment. 

The  development  effor’  consists  of  designing, 
developing,  and  integrating  an  avionics  suite  and  a  gun 
suite,  iiicluding  the  development  of  organizational, 
intermediate,  and  depot  level  support  equipment. 

The  production  estimate  includes  procurement  of.  eleven 
C-130  air  vehicles  and  engines,  conversion  to  the  gunship 
configuration,  delivery  of  organizational,  intermediate,  and 
depot  level  support  equipment,  training  equipment, 
government  furnished  equipment,  and  armament  Th.e 
development  unit  will  be  refurbished  into  productr.oii 
configuration. 
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Advanced  Cruise  Miss i 1 e  f  3 : 3 ) 


The  Advanced  Cruise  Missile  (ACM)  program  improves 
existing  ACM  capability  while  utilising  improved  range  and 
low  observable  capability. 

The  ACM  is  an  ai  rbraathing ,  subsonic,  air:  launched, 
cruise  missile  encompassing  a  basically  cylindrical  fuselage 
with  a  sharp  nose,  chines,  body  RAM,  forward  swept  pcp-ou"; 
wings,  flush  inlet  and  partially  shielded  exhaust. 

The  ACM  sati.sfie3  the  otrategic  Air  Command's  state,  .enr  cf 
need  lor  Increased  range,  improved  survi va.di  1 1 1  y  ,  and 
accuracy.  Pto-Jaction  of  .1461  of  these  missiles  is  planned, 
of  which  120  will  he  a  variar.t  with  a  classified  mission. 

The  vaciaiiL  ccuf ly ur at-ion  is  the  same  as  the  .ACM,  e.Ycept  tor 
provisioning  for  a  heavier  payload. 


The  primary  mission  of  the  ATARS  is  to  provide  near 
real  time  reconnaissance.  ATARS  is  composed  of  a  manned 
reconnaissance  effort  designated  Tactical  Air  Reconnaissance 
System  (TARS) ,  and  an  unmanned  effort  designated  Unmanned 
Air  Reconnaissance  System  (UAR3) .  The  program  will  replace 
existing  reconnaissance  film  sensors  with  electro-optical 
sensors  and  supporting  subsystems  to  provide  near  real  time 
reconnaissance  capability.  TARS  includes  the  TARS  sensor 
upgrade.  The  UARS  will  combine  the  Navy's  Mid  Range 
Remotely  Piloted  Vehicle  with  some  of  the  elements  of  the 
TARS  sensor  suite  and  an  infrared  line  scanner. 


Air  Force  One  (5:2-4) 


The  Air  Force  One  (AF-1)  program  will  purchase  anu 
modify  two  commercial  widebody  Boeing  747  aircraft  to 
replace  current  Boeing  707  aircraft  used  for  presidential 
travel.  The  new  AF-1  aircraft's  mission  is  to  safely  and 
secuL.ly  transport  the  President,  his  family,  staff,  guests 
security  contingent,  and  the  press  to  any  world-wide 
destination.  The  aircraft  will  be  certified  in  accordance 
with  applicable  Federal  .Aviation  Regulat:ons. 

The  development  effort  incorporates  such  tasks  as 
design  engineer’ng,  integrat-on  and  analysis,  and  system 
test  and  certification.  The  procurement  costs  also  include 
training  u£  an  initial  cadre  of  Air  Force  aircrew  personnel 
delivery  of  data,  an  integrated  logistics  support  plan  and 
site  conversion. 
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The  B-IB  Siinulator  System  consists  of  five  weapon 


system  trainers,  two  Mission  Trainers,  six  Cockpit 
Procedures  Trainers,  and  one  Software  Support  Center  to 
support  the  Bl-B  aircraft  The  simulators  will  provide  the 
increased  visual  motion  and  aural  cues  tor  the  ground 
training  of  Strategic  Air  Command  B-IB  aircrew  members. 
Required  training  tasks  include:  mission  rehearsal  training 
for  takeoff  and  landing,  navigation,  air  refueling,  threat 
anal  ysis/cQuntermeasures  ,  low-leve''  peiietr atioi. ,  weapons 
delivery  and  emergency  procedures.  Emphasis  will  be  placed 
on  training  that  cannot  be  accomplished  in  the  aircraft  and 
also  on  integrated  crow  training.  These  tasks  include  those 
related  to  safety  of  flight,  emergency  procedures,  and 
emergency  war  order  rehearsal . 

In  the  30  September  1988  cost  estimate,  the  3-13 
Simulator  System's  risk  due  to  technical  advances  was  rated 
as  difficult  and  the  configuration  design  requirements  was 
considered  to  be  moderately  uncertain.  The  configuration 
uncertainty  of  the  B-IB  Simulator  is  due  to  difficulties  in 
the  defensive  systems  of  the  3-13  aircraft.  The  3-lB 
schedule  has  slipped  two  years  in  both  development  and 
production . 
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G-17 


(10) 


The  purpose  of  the  C-'17  aircraft  is  to  modernize  the 
airlift  iLeet  and  improve  the  United  States'  overall 
capability  to  rapidly  project,  reinforce  and  sustain  corioat 
forces  worldwide.  The  aircraft  will  augment  the  C-5  and 
C-141  in  intertheater  deployment  and  the  C-130  with 
intratheater  operations.  The  C-17  is  capable  of  carrying 
outsize  cargo  over  intertheater  ranges  into  austere 
airfields.  This  introduces  a  direct  deployment  capability 
that  will  significantly  improve  airlift  responsiveness, 
thereby  improving  the  mobility  of  the  general  purpose 
forces . 

In  the  MVt:  estimate  there  were  182  aircraft  in  the 
production  b”y  schedule.  The  1990  Secretary  of  Defense 
directed  Major  Aircraft  Review  resulted  in  the  reduction  of 
the  total  aircraft  buy  for  this  program  from  210  aircraft  in 
the  DE  to  120  aircraft  in  the  CE .  This  reduction  was  in 
anticipation  of  a  25%  increase  in  the  Program  Acquisition 
Unit  Cost. 

Based  on  the  ASD  ECO  model  inputs,  t.he  C-17  effort  is 
considered  to  be  of  low  technical  risk.  The  configuration 
changes  are  due  primarily  to  weight  reduction  made  during 
the  development  process.  The  engineering  change  orders  in 
the  development  effort  totaled  $106.2  million  in  then  year 
dollars.  This  total  was  derived  by  adding  the  amount  of 
engineering  change  proposal  40,  $67.3  million,  and  $38.4, 
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the  dollar  value  estimated  using  ASD ' s  SCO 
Initial  Operational  Capability  schedule  of 
ai  raft  slipped  15  months;  from  June  1993 


model .  The 
the  first  twelve 
to  September 


1994. 
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C-17  Aircrew  Training  System  ( 8 ) 


The  C-17  7»ircrew  Training  System  (ATS)  is  a  •[•arz  of  the 
C-17  Weapon  System  which  emphasizes  task/performance 
oriented  training  and  will  make  effective  use  of  training 
equipment  to  produce  the  required  C-17  aircrew  skills.  It 
IS  based  upon  the  "’5  C-17  missions  defined  in  the  air 
vehicle  specifications  and  is  sited  by  the  student 
throughput  rate  and  aircraft  basing.  It  will  consist  of  the 
following  types  of  training:  initial  qualification,  mission 
qualification,  continuation  training  (proficiency  and 
mission),  and  upgrade  training.  The  basing  of  the  C-17  ATS 
will  include  one  school  .squadron,  five  operational  wing.s 
located  at  separate  Military  Airlift  Command  bases  and  the 
training  base,  two  Air  Force  Reserve  bases,  and  two  Air 
National  Guard  bases. 

Acquisition  of  the  C-17  was  divided  into  two  phases. 
Phase  I  involves  the  initial  training  system  design.  In 
this  phase  three  contractors  were  selected  to  work  on  their 
training  system  design.  Phase  II  consists  of  training 
system  design  co.npletion,  full  scale  development,  proi’uction 
activation,  and  operation  and  support.  One  contractor  was 
selected  to  complete  Phase  II. 
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C-17  Maintenance  Training  Device  (MTD)  ( 9 ) 


The  C-17  MTD  program  is  a  part  of  the  C-17  weapon 
system  that  will  emphasize  task  oriented  training  and 
effecti/e  training  devices  to  produce  the  required  C-17 
maintenance  personnel  skills.  The  goal  is  to  deliver  the 
rtTDs  identified  by  Air  Training  Command  (ATC)  to  each  of 
five  main  operating  bases  to  be  available  120  days  prior  to 
the  delivery  of  the  first  aircraft  at  each  base.  The  MTDs 
will  be  considered  the  prime  source  of  follow-on  training  in 
lieu  of  the  aircraft  to  minimize  the  impact  cf  training 
requirements  on  operational  aircraft.  The  maintenance 
training  courseware  will  be  developed  by  ATC. 

The  development  pnase  of  this  program  requires  the 
design  development,  assemoly,  test,  and  delivery  of  twelve 
distinct  maintenance  training  devices.  It  also  includes 
trai.jing  of  the  initial  cadre  of  Air  Force  instructors 
personnel,  delivery  of  data,  and  the  implementation  of  a 
.•Maintenance  Training  Device  Support  Center. 
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C-20A/B  Source  Selection  (C-SAM;  (11) 

The  Special  Airlift  Mission  (SAM)  provides  worldwide 
transportation  for  the  President  and  Vice  President  of  the 
Unites  States,  cabinet  members,  and  other  high  ranking 
dignitaries  of  the  United  States  and  foreign  governments. 

The  C-140BS  currently  fulfilling  this  mission  are  used  for 
CONUS  and  European  travel  requirements.  The  C-23A/3  is 
being  used  in  a  similar  manner  with  occasional  trips  flown 
worldwide  where  the  number  of  passengers  does  no*,  warrant 
t.  .  use  of  a  larger  aircraft.  The  C-20A/B  will  carry  from 
14-ia  passengers  and  a  crew  of  5  and  is  an  existing  "off- 
the-shelf”  FAA  certified  business  jet. 

There  is  n  Full  Scale  Development  effort  involved  in 
this  buy.  The  buy  schedule  of  this  aircraft  was  accelerated 
by  two  years,  resulting  in  considerable  savings  to  the  Air 
Force.  The  overall  risk  of  the  C-SAM  program  is  considered 
to  be  low.  Primarily,  because  the  aircraft  is  a  commercial 
off-the-shelf  business  jet  with  some  peculiar  gcvernmer; t 
communications  equipment  that  the  contractor  has  previously 
installed  in  the  aircraft  for  other  customers.  The  dollar 
amount  of  the  engineering  change  orders  was  $0.3  million  or 
.12%  of  the  most  probable  cost  estimate. 
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C-130  Aircrew  Training  System  (ATS)  (12) 


The  C-130  ATS  provides  initial  and  mission 
qualification  and  continuation  training  for  all  crew 
positions  on  the  C-130E  and  C-130H  aircraft.  The  program 
requires  the  contractor  to  design,  develop,  produce  and 
operate  the  ATS  with  a  guaranteed  student  being  the  end 
product . 

Government  furnished  equipment  plays  a  very  large  role 
in  this  program.  The  Air  Force  currently  owns  12  C-130 
simulators  that  will  be  turned  over  to  the  contractor  for 
hi.  use.  Additionally,  all  the  spares  and  support  equipment 
unique  to  the  C-130  simulators  and  some  common  support 
equipment  and  spares  will  be  turned  over. 

The  contractor  will  develop  curriculum  for  the  entire 
spectrum  of  training,  from  initial  qual if i :ation  to 
ccrtinuation  training,  for  all  crew  positions.  Twenty-five 
"ourses  will  be  developed  and  delivered.  The  training 
system  support  center  is  the  heart  of  the  ATS.  It 
accomi  ishes  configuration  control  of  all  the  simulators  and 
other  ATS  components,  maintains  documentation,  and  modifies 
simulator  software. 
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Combat  Talon  II  ^ 1 3 ) 

The  Combat  Talon  II  System  is  a  new  production  C~130H 
integrated  with  sophisticated  off-the-shelf  avionics 
systems.  The.se  avionics  enable  the  Combat  Talon  II  aircraft 
to  airland  or  airdrop  heavy  payloads  after  flying  long  range 
at  low  altitude  using  a  high  precision  autonomous  navigation 
system.  Mission  profiles  include  adverse  night  and  combat 
environments.  Being  a  G-130  derivative,  the  aircraft  can 
land  on  and  take  off  from  short  unprepared  fields.  It  is 
a’ so  capable  of  in-flight  refueling  as  a  receiver. 

The  procurement  approach  was  to  award  separate 
contracts  for  the  production  aircraft  and  specialized  Combat 
Talon  II  structural  subsystems  and  another  contractor  for 
the  avionics  integration. 
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Cruise  Missile  Missir g  Control  Aircraft  (14) 

The  Cruise  Missile  Mission  Control  Aircraft  (CMMCA) 
program  is  a  Class  II  modification  of  two  Air  Force  C-18, 
Boeing  707-323CF,  aircraft  currently  owned  by  the  4950th 
Test  Wing.  Upon  completion  of  the  modification,  both 
aircraft  will  have  che  capability  to  receive,  record,  and 
retransmit  telemetry  data.  Each  CMMCA  will  provide  real 
time  telemetry  data  processing  and  display,  and  a  fully 
redundant  Remote  Command  and  Control  System  si.milar  to  the 
CMMCA  Phase  Zero  aircraft.  Finally,  a  government  furnished 
radar  will  provide  the  means  to  track  the  position  of  the 
cruise  missile  while  surveying  the  surrounding  airspace  for 
in^riignt  collision  avoidarice.  Total  syst am  mtagraticn 
will  provide  instantaneous  intormarion  to  on-board  test 
personnel  for  making  critical  in-flight  mi.ssion  anci  safety 
decisions  on  a  real-time  basis.  The  resulting  modified  C-i3 
will  serve  as  the  sole  support  aircraft  for  the  duraticn  of 
a  cruise  missile  test  from  ore-launch  to  missile  recovery; 
thereby,  eliminating  the  need  for  fighter  chase  aircraft  or 
AWACS  aircraft  support. 

Prior  to  C’lMCA  modification  the  aircraft  will  undergo  a 
cockpit  upgrade  from  a  commercial  configuration  to  a 
militarised  configuration.  This  effort  is  inciudect  in  the 
CMMCA  mod:.,  fi  cat  ion  . 
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F-111  Digital  Flight  Control 
System  (DFCS)  Program  (15) 


The  DFCS  is  a  Class  IVA  ...edification  to  correct  an 
existing  safety  deficiency  and  to  improve  reliability, 
maintainability,  and  commonality  by  replacing  the  current 
analog  flight  control  system  with  a  state-of-the-art,  fail 
operational / fai 1  safe  digital  flight  control  system  for  four 
of  the  six  Model  Design  Series  of  t’\e  P-111.  The  program 
originated  a:  a  result  of  recommendations  made  to  HQ  USAF  by 
the  F-lll  System  Safety  Group  and  a  Blue  RiLbon  Panel . 

The  DFCS  modification  consists  of  Group  B  and  Group  A 
changes  to  the  existing  F/FB/EF-111  flight  control  system 
and  associated  modifications  to  training  equipment  and 
3  pport  equipment. 

Technical  risk  is  considered  low,  since  the  technology 
being  employed  throughout  the  Digital  Flight  Control  /stem 
is  mature  and  well  known.  The  risk  in  this  program  is 
primarily  associated  with  the  software  integration  of  the 
DFCS.  There  are  $23,868  million  in  ECOs  in  the  EMD  effort 
and  $55,551  million  in  the  Production  effort.  The 
Development,  Test  and  Evaluation/ Initial  Operability  Test 
and  Evaluation  slipped  one  month,  from  September  1990  to 
October  1990  and  the  completion  of  Low  In.itial  Rate  of 
Production  slipped  three  months  from  October  1991  to  January 
1992  . 
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Low  Altitude  Navigation  and  Targeting 
Infrared  Night  (LANTIRN) 

The  LANTIRN  system  provides  the  Tactical  Air  Forces 
with  the  capability  of  conducting  counter  air  and 
interdiction  at  night  and  under- the-weather  using  F-15E  and 
F-16C/D  Block  40/42  aircraft.  Th^  prime  system  consists  of 
a  navigation  pod  and  a  targeting  pod  used  in  conjunction 
with  the  aircraft's  Head-Up  Display  and  Head-Down  Display. 

The  estimate  covers  all  costs  tor  EMD  of  both  the 
navigation  and  targeting  Pods.  The  development  estimate 
initially  contained  700  pod  pairs,  or  1400  pods.  The 
quantity  has  since  been  reduced  to  1067  pods;  561  navigation 
pods  and  506  targeting  pods.  This  decrease  includes 
deleting  the  A-10  aircraft  eftort  and  a  decrease  in  the 
President's  FY90  Budget  (17:3). 

According  to  an  Independent  Technical  Review  conducted 
on  the  LANTIRN  program,  the  LANTIRN  program  was  considered 
to  be  extremely  complex  (16:54).  The  technical  review  gives 
the  target  recognizer  credit  for  the  bulk  of  the  risk  and 
complexity  inherent  in  the  LANTIRN  program  (16:54).  The 
La:  let  recognizer  uses  immature  technology  which  is  likely 
to  require  extensive  redesign  before  it  is  suitable  to 
perform  the  LANTIRN  mission  (16:54).  In  addition  to  the 
high  complexity  of  the  target  recognizer,  "the  boresighc 
correlator,  the  environmental  control  units,  power  supplies, 
and  the  pods'  weight  and  electrical  power  consumption 
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provide  significant  technical  challenges  to  the  designers" 
(16:54)  . 

The  development  estimate  for  the  LANTIRN  ^.rogram 
originally  had  a  Development  Test  and  Evaluation  (DT&E)/ 
Initial  Operability  Test  and  Evaluation  (lOT&E)  date  of 
September  1984.  This  DT&E/ lOT&E  date  slipped  seventeen 
months  to  March  1986,  primarily  due  to  the  time  needed  to 
test  the  Navigation  Pods  (17:4).  The  production  decision 
for  the  Targeting  Pod  slipped  fifteen  months:  however,  the 
schedule  for  the  Navigation  Pod  was  on  time  (17:4).  For  the 
purpose  of  this  research  the  overall  production  program  will 
be  considered  to  have  slipped  fifteen  months,  despite  the 
on-time  delivery  of  the  navigation  pods.  The  engineering 
change  orders  associated  with  the  production  program  is 
$38.4  million  in  hen  year  dollars. 
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Low  Altitude  Navigation  and  Target: n 


,  i.  H  I.  r  u 

Night  (L^NTIRN)  Simulator  (18) 


The  LANTIRN  Simulator  will  provide  a  simulation  of  the 
LANTIRN  Navigation  and  Targeting  Pods.  It  will  be 
integrated  with  an  F-16  Operational  Flight  Trainer  to  enable 
pilots  to  train  in  the  LANTIRN  mission.  The  LANTIRN 
Simulator  will  consist  of  a  commercially  available 
computational  system  and  an  image  generation  system.  The 
computational  system  will  process  dynamic  forward  looking 
infrared  radar  image  simulation  with  adequate  detail  and 
reality  to  support  low  level  navigation  and  target 
identification. 


SRAM  II  (19) 


The  Short  Range  Attack  Missile  (SRAM)  II  vehicle  is  a 
rocket  powered  missile  optimized  for  an  Ai r- to-Surf ace 
nuclear  role.  The  fundamental  requirement  is  to  develop  and 
deploy  a  state-of -the-ar c  SRAM  II  as  a  replacement  for  the 
aging  SRAM  A.  SRAM  II  will  improve  the  penetrating  bombers' 
total  effectiveness  and  employment  flexibility,  and  will 
challenge  the  defensive  threat  with  a  highly  survivable 
weapon  system. 

The  initial  production  quantity  of  1633  units  was 
reduced  by  933  to  700  units. 

The  SRAM  II  program  is  considered  to  have  high 
technical  risk.  The  dollar  amount  of  the  ECOs  also  support 
this  claim.  There  are  $97.2  million  in  ECOs  linked  to  the 
development  effort  and  $58.2  million  in  the  production 
effort.  The  percentage  of  ECOs  to  MFC  is  9%  in  the 
development  phase  and  4.4%  in  the  production  phase. 

There  is  a  two  year  slip  in  the  EMD  schedule  which  has  also 
resulted  in  a  two  yeT  slip  in  the  production  schedule.  The 
revised  production  schedule  is  to  begin  in  FY93. 
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Appendix  B:  C-I7  Quantity  Adjustment 


The  recurring  cost  profile  from  the  December  1990  SAR 
was  used  to  develop  a  cost  improvement  curve  of  the  current 
estimate.  The  recurring  base  year  dollars  and  the  buy 
schedule  taken  from  the  SAR  were  input  in  into  ICPRO  to 
generate  the  cost  improvement  curve  for  the  arrent  quantity 
buy.  The  current  buy  cheduie  indicates  that  there  will  be 
a  break  :n  production  fiscal  year  1991.  However,  due  to 
the  production  schedule  slip,  the  one  year  break  will  not 
have  the  effect  of  a  true  break  (38).  For  this  reason,  the 
cost  improvement  curve  was  calculated  assuming  no  break  in 
the  production  schedule  and  combining  the  advance  buy  in 
fiscal  year  1991  with  the  fiscal  year  1992  production  buy. 
The  results  of  the  ICPRO  run  yielded  a  slope  of  80.81%  and  a 
theoretical  unit  one  (T,)  cost  of  S314.499  million.  This 
slope  and  T-. ,  were  input  into  ICPRO  using  the  buy  schedule 
for  132  units.  The  output  from  this  run  was  the  recum.ng 
costs  by  fiscal  year. 

To  derive  the  ad:iusted  funding  summary,  the  recur  ri.ng 
cost  and  non  recurring  costs  of  the  current  estimate  were 
subtracted  from  the  total  cost  to  obtain  the  dollar  amount 
of  peculiar  support  equipment,  data,  and  other  items 
essential  to  the  program  (heretofore  referred  to  simply  as 
PSE,  since  PSE  makes  up  the  bulk  of  this  figure).  PSE  as  a 
percentage  of  the  recurring  cost  was  derived  by  dividing  the 
difference  obtained  above  by  the  recurring  costs.  The 
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★  * 

*  T  C  L  0  T 

*  * 

*  UNIT  CURVE  THEORY  * 

★  * 

*  VERSIONONE  * 

★ 


TABLE  I 


USER 

INPUT'S 

LOT 

FIRST 

LAST 

AVERAGE 

# 

UNIT 

UNIT 

LOT  COST 

1 

1.00 

2.00 

202.10 

3.00 

6.00 

135.38 

3 

7.00 

10.00 

148.68 

4 

11.00 

16.00 

184.20 

5 

17.00 

28.00 

145.00 

6 

29.00 

40.00 

138.52 

7 

41.00 

58.00 

92.88 

8 

59.00 

76.00 

80.27 

9 

77.00 

94.00 

78.44 

10 

95.00 

112.00 

74.32 

11 

113.00 

120.00 

61.19 

TABLE  II 

LErtSV  SQUARES  ANALYSIS 


uJKBUTED  VALUE  OF  FIRST  UNIT  —  A  .  315.499 
REGRESSION  SLOPE  COEFFICIENT  —  B  -  -0.307 
REGRESSION  SLOPE  PERCENTAGE  —  S  -  80.821 
COEFFICIEtJT  OF  CORRELATION  —  R  -  -0.878 
COEFFICIElJT  OF  DETER.NINATION  —  R-SQ  -  0.771 


TABLE  III 


REGRESSION  LINE 

COMP.ARISON 

COMPUTED  LOT 
MIDPOINT 

CALCULATED 
Y  AT  X 

ACTUAL 

V 

PERCENT. 

OIFFEREl 

1.388 

285.245 

202.100 

-29.149 

‘..311 

201 . 398 

135.375 

-32.782 

3.403 

154.065 

148.675 

-9.380 

13.357 

1-2.292 

184. 2C0 

29.-52 

22. 148 

121.819 

1-5.000 

19.029 

34.273 

106.523 

138.525 

30.036 

^+9 .  1^2 

95.36*. 

92.378 

-Z .  f?07 

67.233 

36.608 

30.267 

-  ^  .  3  Z 1 

85.294 

30. 5C5 

78. -..0 

-2.5t)5 

103.320 

75.398 

74.322 

-2.0'6 

116.-71 

73 , 257 

61.138 

-16.362 
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